In a formal ceremony attended by top brass and ministers at the presidential palace, Yudhoyono swore in 52-year-old Lieutenant General Joko Santoso as army chief, replacing an outspoken and controversial nationalist.
The shakeup also saw Rear Marshall Slamet Sugianto appointed head of the navy and Rear Marshall Joko Suyanto put in charge of the air force.
The reshuffle was seen by analysts as an attempt by Yudhoyono, himself a former general, to assert his influence over the military.
A new overall chief of the armed forces is expected to be put forward in the next three months.
The shakeup's most notable departure was that of army chief of staff Ryamizard Ryacudu, a brash and outspoken general who has claimed the country is riddled with spies bent on destabilisation.