First of all, you need to tell us what OS you are using. We can tell you the exact System Preference pane you need to go to. (In Yosemite, it is called "Language and Region", but for earlier versions of the Mac OS (not sure how far back), it is called something else).
Secondly, does this occur "everywhere"? That is, even when you use a Word Processing program like Word or TextEdit, or a spreadsheet like Excel?