I use a grid notebook. It is great for drawing quick designs of boards and cards, and allows you to write lists and columns of things very easily. I have quite a thick yet small one, so I can take it around with me.
When I first have a game idea, I like to draw out some of the components that are in my head, just to get a feel for how the game might seem. Often board or card ideas come first. As I go I jot down all the little rule ideas that pop into my head. Over time I develop these and crystalise ideas in the book until I am ready to make a pototype. Here is an early page of notes for a game called 'Nerd Fighting League'. I doubt this will every get made, but a few elements from this have made it into a newer design of mine. I have plenty of pages like this one which didn't quite capture me enough to go on with the design process.
When a game design is some way along, I also use the notebook to work out many of the details for the game. For example, on this page I was working out the balance of card numbers for Archaeology. I also started playing around with specific design ideas for the board and cards.
When a game design is pretty close to finished I also use the notbeook for plotting our the nitty-gritty of production, costs etc. Here I am writing out the different options for the components I might produce for a game. I have found lists like this very useful in comparing prices and different costing options. This sort of process has helped me make decisions about board size, numbers of components and plenty of other things.
Hope that was interesting! I'd be keen to hear how others keep track of all their ideas..