The phrase meant that Harding wanted to return to pre-WWI conditions. He wanted things to go back to the "good old days," before the war came and took so many lives.
The world was out of his control. There were too many things happening like
1. parties and flappers.
2. bank loans that were not even held up by hot air balloons
3. The work force was changing. Women were not content to stay home and raise children.l