President James Madison was all for the war, but many were against it:
Few suffered more than the group that defended Alexander Contee Hanson’s right to publish the flamboyantly anti-war Federal Republican paper in Baltimore in June 1812. A heavily-armed group defended the publishing house against a riotous Baltimore crowd that boasted an artillery piece manned by none other than the editor of the rival Sun newspaper. When the affair ended, one of the defenders was dead and eleven more were physically broken following hours of physical torture. These were hardly anti-American radicals. Among the severely wounded was Henry “Light-Horse Harry” Lee: Revolutionary War hero, former governor of Virginia, and father to the Confederate army general, Robert E. Lee.
Nothing like a mob to bring clarity to a situation.
So Light-Horse Harry Lee fought in the Revolution, then later fought for a friend who was against the War of 1812, and then later still his son became the super-general for the secessionists in the Civil War. Sides shifted quickly back in those days.