While the potatoes are still warm, mix in the egg yolks 1 at a time using a wooden spoon or rubber spatula, making sure each yolk is completely incorporated before adding the next. Add the butter, cream, measured salt, pepper, and nutmeg and stir until the butter has melted and the mixture is smooth.