Born the son of Brigadier The Honourable William Fraser (1890-1964) DSO MC, who had been the military attach in Paris when the Second World War begun, David Fraser was educated at Eton College and Christ Church College, Oxford. He left school to enlist at earliest opportunity after the Second World War begun, and joined his father's regiment, the Grenadier Guards in 1940, serving for much of the War with the Guards Armoured Division, later in North West Europe, ending the war in the rank of Major.
He was appointed General Officer Commanding 4th Division in 1969, Assistant Chief of Defence Staff (Policy) in 1971 and Vice Chief of the Imperial General Staff in 1973. He went on to be British Military Representative to NATO in 1975, and Commandant of the Royal College of Defence Studies in 1977 before retiring in 1980.