Take out a dollar bill (United States currency, that is). Look at the back. On the left side, granted as much space as the American eagle symbol on the right, is a seeing eye and a pyramid, placed there for no apparent reason. But for those in the know, the eye above the pyramid is a Masonic symbol, produced by a secret society which has influenced American history from its beginnings. In Masonic lore, the pyramid symbol is known as a sign of the eye of God watching over humanity.

The Masons have been both criticized and praised for their influential role in U.S. history.

George Washington reached the top level of the Masons on August 4, 1753, securing the leadership of the influential lodge in Alexandria, Virginia. Washington was not alone among the founding founders; some scholars say as many as twenty-one signers of the Declaration of Independence were Masons. Many historians note that the Constitution and the Bill of Rights both seem to be heavily influenced by the Masonic “civil religion,” which focuses on freedom, free enterprise, and a limited role for the state.

In Europe, the Masons were known for plotting against royal governments. In America, they became known for promoting Republican virtues of self-government.

Masonic thought influenced American history: the Masons were opposed to the claims of royalty—a strong influence on the development of the American revolt against Britain which culminated in the Revolutionary War. They were also known for their opposition to the Catholic Church, another international organization that competed for allegiance.

Today’s Masonic lodges in the U.S. have a largely benign public image, seen as a place for smalltown businessmen (the order is limited to men) to engage in social gatherings, networking, and opportunities for charity. But the group, with its secret symbols and handshakes, was not always so harmless.

The United States Masons (also known as Freemasons) originated in England and became a popular association for leading colonials after the first American lodge was founded in Boston in 1733. Masonic brothers pledged to support one another and provide sanctuary if needed. The fraternity embodied European Enlightenment ideals of liberty, autonomy, and God as envisioned by Deist philosophers as a Creator who largely left humanity alone.

Those theological views created friction with established Christian churches, particularly Catholics and Lutherans. While the Masons captured the allegiance of much of the early Republic’s elite, the group did fall under widespread suspicion. The William Morgan affair of 1826—when a former Mason broke ranks and promised to  expose the group’s secrets—threatened its demise. Morgan was allegedly abducted and presumed killed by Masons, and the scandal proved a low point in the public image of the fraternal order.

The anti-Mason backlash grew. Abolitionists like John Brown railed against the often pro-slavery Masons. Prominent figures including John Quincy Adams, a former president and former Mason, and publisher Horace Greeley joined in the widespread castigation. Future president Millard Fillmore called Masonic orders nothing better than “organized treason.” In 1832, an anti-Masonic party ran a one-issue candidate for president. He captured Vermont’s electoral votes.

American Masons were not above engaging in controversial foreign adventures. In 1850 a contingent of American Masons and Mexican War veterans invaded Cuba to foment a rebellion against the Spanish crown. The group failed to gain a foothold and retreated after suffering heavy casualties. Its leaders were later tried in New Orleans for violating U.S. neutrality laws.

The group’s long-term fraternalism and secrecy has traditionally served as a vehicle of exclusion, not inclusion. Today, its reputation is buttressed by an affilation with the Shriners, a related fraternal group noted for its charity and health work. The Masons’ revolutionary and sometimes violent past now serves as a kind of historic footnote as the order established itself as a placid participant in the American social fabric. Even with its controversial past, it’s hard to imagine the Masonic order serving as a contemporary hotbed of violent insurrection.