Lee has been brought in by Tottenham to help persuade the Olympic Park Legacy Company (OPLC) to back their proposed 250 million pound (394 million dollars) move to the Stratford site of the main stadium for the 2012 London Games.
The proposal is controversial both with Spurs fans, who don't want the club to leave north London and with Premier League rivals West Ham, who want to move to an Olympic site close to their existing Upton Park ground.
When London won the right to stage the 2012 Games, officials led by former double Olympic 1500m champion Coe promised the new stadium would leave a permanent legacy for athletics in the British capital.
West Ham have said they would keep the running track as part of a 60,000-capacity stadium.
Tottenham, by contrast, would demolish the track and contribute to athletics by redeveloping the existing facilities at Crystal Palace in south London.
Coe has said he feels a "moral obligation" to support West Ham in their plan to keep the Olympic Stadium as a multi-sports venue while Lee -- a former Hammers board member -- is now advising Spurs in their rival bid.
There are growing concerns over whether West Ham, who face relegation from the Premier League, can maintain the stadium and indeed whether London needs a new permanent athletics track once the Games are over.
Meanwhile, Lee said it was tough for football and athletics to co-exist.
"If you're not in a great football stadium it affects your experience and your willingness to come week in, week out," he told the Guardian.
"The long-term future of athletics is also important. And we know that football and athletics don't work as a combination.
"Bayern Munich didn't have a great time in their Olympic stadium and athletics hasn't really worked there since the (1972) Games. Espanyol were also happy to move out of Barcelona's Olympic Stadium.
"I genuinely think London has an opportunity to create the greatest Olympic Park ever," added Lee, who has also advised Rio de Janeiro and Qatar on their successful bids to stage the 2016 Olympics and 2022 football World Cup respectively
"I'm not saying it's a simple decision. But if you duck difficult decisions when it comes to the legacy of these great sporting events you're in danger of creating a white elephant.
West Ham vice-chairman Karren Brady said Spurs would be guilty of a "corporate crime" if the Olympic Stadium was torn down.
She added Britain, looking to restore its standing amongst global sports chiefs following England's failed bid to stage the 2018 football World Cup, would also suffer a huge blow to its reputation if Spurs got the go-ahead.
"On July 6 2005, (when Britain won the right to host the 2012 Olympics) a promise was made in the Queen's name," she said.
"We believe in that promise and we believe in legacy.
"It's important for the UK's credibility as a sporting nation - especially in the wake of the 2018 FIFA World Cup disappointment - to keep that promise."
Last month the OLPC delayed its planned announcement on their preferred bidder for the Olympic Stadium and they have yet to set a new date.
Olympic ally opposes Coe over Stadium future
Advertisement