The IOC executive board decided to retain modern pentathlon - the event considered most at risk - and remove wrestling instead from its list of 25 "core sports."
The IOC board acted after reviewing the 26 sports on the current Olympic program. Eliminating one sport allows the International Olympic Committee to add a new sport to the program later this year.
Wrestling, which combines freestyle and Greco-Roman events, goes back to the inaugural modern Olympics in Athens in 1896.
"This is a process of renewing and renovating the program for the Olympics," IOC spokesman Mark Adams said. "In the view of the executive board, this was the best program for the Olympic Games in 2020. It's not a case of what's wrong with wrestling, it is what's right with the 25 core sports."
Adams said the decision was made by secret ballot over several rounds, with members voting each time on which sport should not be included in the core group. IOC President Jacques Rogge did not vote.
Wrestling was voted out from a final group that also included modern pentathlon, taekwondo and field hockey, officials familiar with the vote told The Associated Press. They spoke on condition of anonymity because the voting details were not made public.
The board voted after reviewing a report by the IOC program commission report that analyzed 39 criteria, including television ratings, ticket sales, anti-doping policy and global participation and popularity. With no official rankings or recommendations contained in the report, the final decision by the 15-member board was also subject to political, emotional and sentimental factors.
The international wrestling federation, known by the French acronym FILA, is headed by Raphael Martinetti and is based in Corsier-sur-Vevey, Switzerland. Calls to the federation for comment were not immediately returned.
Wrestling featured 344 athletes competing in 11 medal events in freestyle and seven in Greco-Roman at last year's London Olympics. Women's wrestling was added to the Olympics at the 2004 Athens Games.
Wrestling will now join seven other sports in applying for inclusion in 2020. The others are a combined bid from baseball and softball, karate, squash, roller sports, sport climbing, wakeboarding and wushu. They will be vying for a single opening in 2020.
The IOC executive board will meet in May in St. Petersburg, Russia, to decide which sport or sports to propose for 2020 inclusion. The final vote will be made at the IOC session, or general assembly, in September in Buenos Aires, Argentina.
It is extremely unlikely that wrestling would be voted back in so soon after being removed by the executive board.
"Today's decision is not final," Adams said. "The session is sovereign and the session will make the final decision."
The last sports removed from the Olympics were baseball and softball, voted out by the IOC in 2005 and off the program since the 2008 Beijing Games. Golf and rugby will be joining the program at the 2016 Games in Rio de Janeiro.
Previously considered under the closest scrutiny was modern pentathlon, which has been on the Olympic program since the 1912 Stockholm Games. It was created by French baron Pierre de Coubertin, the founder of the modern Olympic movement, and combines fencing, horse riding, swimming, running and shooting.
Klaus Schormann, president of governing body UIPM, lobbied hard to protect his sport's Olympic status and it paid off in the end.
"We have promised things and we have delivered," he said after Tuesday's decision. "That gives me a great feeling. It also gives me new energy to develop our sport further and never give up."
Modern pentathlon also benefited from the work of Juan Antonio Samaranch Jr., the son of the former IOC president who is a UIPM vice president and member of the IOC board.
"We were considered weak in some of the scores in the program commission report but strong in others," Samaranch told the AP. "We played our cards to the best of our ability and stressed the positives. Tradition is one of our strongest assets, but we are also a multi-sport discipline that produces very complete people."