Grammy chief Neil Portnow has cancelled his speaking engagement at a live music conference in Los Angeles after his poorly judged comment that female artists and executives need to “step up” if they want to make it.

Portnow was scheduled to talk at the Pollstar Live! Conference this Thursday on a session titled, “Behind the Grammy Awards Show” with the event’s executive producer Ken Ehrlich and Grammy Museum’s Scott Goldman. But as Pollstar reports, Portnow has backed out of the panel and a replacement speaker hasn’t been announced.

Portnow, the president and CEO of the Recording Academy, has faced withering criticism following his remarks during a media gathering backstage at the Jan. 28 Grammy Awards.

In response to queries about why Lorde, the only woman nominated for Album of the Year, was conspicuously absent from the lineup of Grammy performers, Ehrlich remarked, “These shows are always a matter of choices, and we know we have a box and the box gets full and filled up. She had a great album, album of the year is a big honor, but there’s no way we can really deal with everybody. Sometimes people get left out that shouldn’t, but on the other hand, we did the best we can to make sure that it’s a representative and balanced show.”

Portnow chimed in with his controversial “step up” comment, which he has since conceded “wasn’t as articulate” as he should have been.

The industry honcho attempted to explain his words in a statement issued Jan. 30. “Sunday night, I was asked a question about the lack of female artist representation in certain categories of this year’s Grammy Awards,” according to the statement. “Regrettably, I used two words, ‘step up,’ that, when taken out of context, do not convey my beliefs and the point I was trying to make.”

Female artists and executives haven’t let him get away so easily. Fiona Apple staged her own protest by wearing a T-shirt splashed with the words “Kneel Portnow” during a performance with Garbage’s Shirley Manson at the Girlschool L.A. music fest.

Last week, female execs from across the music industry signed a letter calling for his resignation, arguing his message was “wrong and insulting and, at its core, oblivious to the vast body of work created by and with women.” Another, more recent letter, signed by six female music industry leaders calls for an overhaul of the  Recording Academy (or NARAS).

Portnow’s comments are “not a reflection of being ‘inarticulate’ in a single interview. They are, unfortunately emblematic of a much larger issue with the NARAS organization as a whole on the broader set of inclusion issues across all demographics – from the make-up of the voting membership and its transparency, to production of the show, to the organization’s hiring practices and more,” reads an extract of the letter, signed by Michele Anthony, Jody Gerson, Julie Greenwald, Sylvia Rhone, Julie Swidler and Desiree Perez. “To be clear, if NARAS seeks to reflect music’s diverse community then it must ‘step up’ and be accountable to it.”

Read the letter here at MBW.