I was going to try to upgrade to a smart thermostat but the sites I saw online said thermostats always have an R or RC wire. This is a 1920 house with oil furnace (getting very expensive). The last time I tried to add a smart thermostat to a non-C wire unit at our old place I ended up having to give an HVAC company a whole paycheck.
The pictured thermostat clearly lacks an R(h)/RC. So, the answer to your question is; no, not all thermostats have R(h)/RC.
This blog post explains two wire thermostats pretty well.
As you will learn in the blog post, your current wiring cannot utilize a thermostat that requires the R/RC wires for power. The thermostat for your pictured application would need to be self(battery) powered.