It's pretty easy to get to: you go to Victoria Rail Station, and take the train to Bromley South. Trains run rather frequently. Once at Bromley South, you walk out of the station and up the street to the right a bit, and catch the 146 Downe bus. Your
Oyster card works for the train
and the bus - very convenient. The last stop is Downe Church, and then it's a short walk up Luxted Road.
The walk along the road is a bit unpleasant, since it is narrow and there is no sidewalk, so here's a tip: walk up until you see a sign on the left of the road that indicates the footpath to "Cudham". Follow this footpath until there is a sign indicating "Down House" to the right.
Inside you will find exhibits about Darwin's life and work, including his study and library. Outside, you find his gardens, and the famous Sandwalk. Not to be missed if you are in the London area!