Are you a privacy obsessed?
You can do everything in your power to avoid being tracked through your phone, such as change OS, avoid Google's services, only use open-source apps from F-Droid, turn off WiFi and location services but…there is one thing that is majorly overlooked.

Your SIM card's has ability to track you

It has a unique ID that is likely tied to your identity.
It authenticates you to cell towers wherever you go: this means that your provider has access to your location at all times through triangulation:

And it doesn't matter if you switch off mobile data, it will still connect to be able to receive phone calls and messages.

Some goodies from Defcon21

Furthermore, SIM cards also have the ability to accept remote commands, download apps on request by your provider, interface with hardware components by going underneath the OS and more, according with this talk by Karl Koscher & Eric Butler, presented during DefCon21:

SIM cards can do more than just authenticate your phone with your carrier. Small apps can be installed and run directly on the SIM separate from and without knowledge of the phone OS. Although SIM Applications are common in many parts of the world, they are mostly unknown in the U.S. and the closed nature of the ecosystem makes it difficult for hobbyists to find information and experiment.

This talk, based on our experience building SIM apps for the Toorcamp GSM network, explains what (U)SIM Toolkit Applications are, how they work, and how to develop them. We will explain the various pieces of technology involved, including the Java Card standard, which lets you write smart card applications using a subset of Java, and the GlobalPlatform standard, which is used to load and manage applications on a card. We will also talk about how these applications can be silently loaded, updated, and interacted with remotely over-the-air.