Pee Wee Ellis, a saxophonist, was an arranger and composer who fused jazz, funk and soul as the musical director for James Brown and Van Morrison.

Photo: Pee Wee Ellis performing in Rochester, N.Y., in 1958 with Ron Carter on bass. Mr. Ellis bridged jazz and funk, and brought his fusion sensibility to James Brown’s band as an arranger and composer.Credit…Paul Hoeffler/Redferns, Getty Images (cropped. full image below)

27 September 2021 | Jon Pareles | New York Times | with additional photos and video

Alfred (Pee Wee) Ellis, a saxophonist, arranger and composer who fused jazz, funk and soul as the musical director for James Brown and Van Morrison, died on Thursday. He was 80.

The cause was “complications with his heart,” his Facebook page said. It did not say where he died; he lived in Dorset County, England.

Mr. Ellis also performed, arranged and recorded extensively with his own jazz groups, in funk bands with fellow James Brown alumni and as a sideman for a broad array of musicians in jazz, R&B, pop, rock and African music. And his association with Mr. Morrison stretched across two decades.

Mr. Ellis shared credit with Mr. Brown for writing 26 songs performed by Mr. Brown, including “Cold Sweat” and “Say It Loud — I’m Black and I’m Proud.”

He had a collaborative temperament that allowed him to get along with demanding performers like Mr. Brown, Mr. Morrison, Esther Phillips and the rock drummer Ginger Baker. “I’m not hard to get along with — and I’m a good mediator,” he said in a 2020 interview with The American magazine. “All their problems were their problems, not mine.”

Alfred James Ellis was born on April 21, 1941, in Bradenton, Fla. He started playing piano, clarinet and saxophone as a youth, joining the marching band in junior high school. The family moved to Lubbock, Texas, in 1949 after his mother had married Ezell Ellis, who managed local musicians. Those musicians gave Alfred, who was a skinny child, his nickname, Pee Wee.

Ezell Ellis was stabbed to death in a Texas club in 1955; a white woman had insisted on dancing with him, and the killer was infuriated at seeing an interracial couple.

The family moved to Rochester, N.Y., when Alfred was a teenager, and he played jazz in high school groups and in clubs. He also spent time in New York City and studied at the Manhattan School of Music. He made his first recordings as a sideman.

One day, in 1957, he was retrieving his saxophone from a repair shop when he ran into the jazz titan Sonny Rollins on Broadway and boldly asked him for lessons. Mr. Rollins agreed, and Mr. Ellis began making weekly trips to New York City to study with him. In a 2014 interview for the magazine Neon Nettle, Mr. Ellis likened working with Mr. Rollins to being “a sponge in deep water.”

After high school he moved to Miami and became a full-time musician. Members of Mr. Brown’s band saw him performing at a motel there in 1965, and soon afterward he was hired to join the band. In a few months Mr. Ellis had become Mr. Brown’s musical director, writing arrangements and teaching them to the band.

After a show at the Apollo Theater in Harlem, Mr. Brown summoned Mr. Ellis with an idea for a bass line. Then, in the band bus on the way to Cincinnati, Mr. Ellis constructed the rest of the music for what became “Cold Sweat,” a syncopated vamp with a two-note horn line that echoed Miles Davis’s “So What.”

Fiercely polyrhythmic and untethered from blues or pop-song forms, the song became a cornerstone of funk. “‘Cold Sweat’ deeply affected the musicians I knew,” the producer Jerry Wexler said in the liner notes to “Star Time,” a James Brown boxed set. “It just freaked them out. For a time, no one could get a handle on what to do next.”

Mr. Brown and Mr. Ellis wrote “Say It Loud — I’m Black and I’m Proud,” another funk milestone, in response to the murder of the Rev. Dr. Martin Luther King Jr. in 1968 and the subsequent summer of racial unrest.

“It was a music that heralded a new attitude,” Mr. Ellis said in a 2020 interview with Jazzwise magazine, “a new and distinctive Black culture, of street culture finding confidence and popularity outside and alongside the establishment. Sweeping into mainstream consciousness during the civil rights movement was unlike anything people had heard, and its positive energy united a new generation making them proud of their music, fashion and political tastes.”

But relentless touring and recording with the James Brown band was grueling, and as the 1960s ended Mr. Ellis decided to return to jazz. In the 1970s he arranged and conducted the music for full albums by George Benson and Johnny Hammond; he also recorded with Esther Phillips, Leon Thomas, Hank Crawford, Shirley Scott, Sonny Stitt and Dave Liebman. He released his first full album as a leader, “Home in the Country,” in 1977.

Mr. Ellis was invited to do horn arrangements for Van Morrison’s 1979 album, “Into the Music,” starting a lasting relationship. He appeared on Mr. Morrison’s albums for the next 20 years, and had stints as the musical director for Mr. Morrison in the 1980s and 1990s.

In the ’90s and 2000s Mr. Ellis rejoined the saxophonist Maceo Parker and the trombonist Fred Wesley, bandmates from his years with Mr. Brown, to perform and make albums under various names, including the J.B. Horns and the J.B.’s Reunion.

He led his own group, the Pee Wee Ellis Assembly, and made more than a dozen jazz albums as a leader. His touring projects included a stint in the 2010s with a quartet led by Mr. Baker, the drummer from Cream, and “Still Black Still Proud,” a James Brown tribute featuring African musicians.

He also played sessions for, among many others, De La Soul, 10,000 Maniacs, Walter Wolfman Washington, Poncho Sanchez, Oumou Sangaré, Toumani Diabaté, Cheikh Lo and Ali Farka Touré. (Information on his survivors was not immediately available.)

Mr. Ellis told The American that he was happiest when collaborating. “Part of the magic,” he said, “is joining forces and making something happen from nowhere.”