Prior to subdivision in 1908, the area was a paddock owned by the Lemprière family, and was in use as a polo ground. The Lemprières were a prominent Caulfield family with several members serving on the Caulfield Council.
The suburb is a mostly residential area with a strip shopping centre on Gardenvale Road. The suburb is bounded by Nepean Highway to the west, Elster Avenue to the north, Kooyong Road to the east and North Road to the south, and the Elster Creek flows through the suburb before entering the Elwood Canal.
The name Gardenvale is also used as a locality name with less definite boundaries. The railway station, built in the 19th century as part of the Rosstown Railway, was located a few blocks further North on the corner of Riddell Parade and Clarence Street in what is now officially part of Elsternwick. A section of Brighton North on the Western side of the Nepean Highway is also usually called Gardenvale, while postal addresses are officially Brighton or Brighton North.