The Silver Greys, also known as Union Whigs, were a conservative faction of the Whig Party which was born from the Compromise of 1850 emerged as a major faction during the 1852 presidential election, supporting Millard Fillmore and Daniel Webster. The Silver Greys, nicknamed because of their domination by old traditionalist Whigs, supported the Compromise of 1850 and the quieting of sectional issues as a means of retaining political power; William H. Seward's faction of the Whig Party rejected the Silver Greys' pro-Compromise views and nativism. In 1852, Whig presidential hopefuls Webster and Fillmore each took as their platform the Union and the finality of the Compromise. Webster won the support of New England and the large Eastern cities, while Fillmore was overwhelmingly the choice of Southern Whigs. The Whigs' nomination of the moderate Winfield Scott for President in 1852 angered the Silver Greys, as Scott lacked enthusiasm for the Compromise of 1850 and also pandered to Irish and German Catholic immigrant voters at the expense of the staunchly anti-Catholic Presbyterian and Methodist Whigs and the Whigs' traditional allies, the Know Nothings of Pennsylvania. In Ohio, many native-born Whigs voted for Pierce, while, in Philadelphia, ultra-Whig Protestants voted for Pierce due to their hatred of the Irish immigrant community. After the Whig defeat at the 1852 presidential election, the Sewardites and Silver Greys called on the Whig Party to purge the rival faction, with the Sewardites calling the Silver Greys "dead weight". A number of conservative Whigs spoke favorably of a new Silver Grey-conservative Democratic coalition.
In New York, a state in which both the national Whig and Democratic parties were torn apart by factionalism, conservativse nominated a pro-Compromise Union ticket of both Whigs and Democrats in 1850. In New York City in 1852, Silver Greys made up a large portion of the non-voting and bolting Whigs. During the 1853 elections, several Silver Greys allied with the Democratic Hunkers' conservative "Hardshell" faction in forming conservative coalitions at the county level, while remining loyal to the national party at the state level.
Following the Kansas-Nebraska Act of 1854, Northern conservatives were angered at the needless revival of sectional animosities. In New York, both the national and sectional Whigs disapproved of the repeal of the Missouri Compromise, healing the divisions within the New York Whig Party. Many Silver Greys were reluctant to quit the Whig Party and form an anti-Nebraska movement, opposing the Thurlow Weed-inspired Whig ticket in the fall of 1854. The Silver Greys later formed an alliance with the Know Nothings in the New York state legislature to prevent Governor Seward's re-election to the US Senate and overthrow the Seward, Weed, and Greeley faction. The Silver Greys predicted that their alliance with the Know Nothings could sweep Sewardism and "Political Catholicism" off the face of the earth. While Silver Grey Know Nothings united with Free Soil natvists in condemning the Kansas-Nebraska Act, the New York Evening Post argued that the idea that this movement could be easily converted into a conservative Union party was a "delusion".
At the 1854 New York gubernatorial election, disaffected Silver Greys took control of the state Know Nothing organization and nominated the conservative perennial candidate Daniel Ullman for Governor. Where they were organized, the Know Nothings swept away the established parties, winning 26.03% of the vote; while Whig Myron H. Clark narrowly won the election with 33.38% of the vote, the election was a long-term defeat for the Whigs, whose traditional voters deserted the party in droves. Many of Clark's votes were based on his pro-temperance views; he suffered massive losses among native-born voters, ran behind Ullman in a number of Whig strongholds, and picked up unexpected support in northern counties due to his anti-Nebraska position. The 1854 gubernatorial election destroyed the old Whig party organization, with nativism sealing the party's fate. The Silver Grey faction thus merged into the Know Nothings, while Seward's faction - backed by the anti-Nebraska movement, Free Soilers, the Anti-Rent Party, and the temperance movement, would eventually form the core of the state's Republican Party.