Preheat oven to 350°F (175°C). Grease and flour a 9x13 inch baking pan.

In a large bowl, whisk together flour, baking soda, salt, cinnamon, nutmeg, ginger, and cloves.

In a separate bowl, whisk together granulated sugar, brown sugar, vegetable oil, and eggs until well combined.

Add the pumpkin puree and milk to the wet ingredients and mix until smooth.

Gradually add the dry ingredients to the wet ingredients, mixing until just combined. Do not overmix.

Pour the batter into the prepared baking pan and bake for 30-35 minutes, or until a toothpick inserted into the center comes out clean.

Let the cake cool completely in the pan, then cut into 1-inch cubes.

In a medium bowl, whisk together egg yolks, granulated sugar, and cornstarch until smooth.

In a saucepan, heat the milk over medium heat until it is steaming but not boiling.

Slowly pour about half of the hot milk into the egg yolk mixture, whisking constantly to temper the eggs.

Pour the tempered egg mixture back into the saucepan with the remaining milk. Cook over medium-low heat, stirring constantly, until the custard thickens enough to coat the back of a spoon.

Remove from heat and stir in the vanilla extract.

Pour the custard into a clean bowl, cover with plastic wrap directly on the surface to prevent a skin from forming, and refrigerate until completely chilled.

In a large bowl, whip the heavy cream with an electric mixer on medium-high speed until soft peaks form.

Add the powdered sugar and vanilla extract, and continue whipping until stiff peaks form.

In a large trifle dish or individual serving glasses, layer about one-third of the pumpkin cake cubes.

Spoon about one-third of the chilled custard over the cake layer.

Spread about one-third of the whipped cream over the custard.

Repeat the layers two more times, ending with a layer of whipped cream.

Garnish with chopped nuts and drizzle with caramel sauce, if desired.

Chill the trifle for at least 1 hour before serving to allow the flavors to meld.