Picking an agent:
Shop around for an agent who understands your needs and has good knowledge of the local area. They should know any regulations in the area, and distances to amenities.
Legal regulations:
Consider what you want to do with a property. You may want to keep livestock, build on it, or subdivide. Rural legal restrictions can often be stricter than in the city, so decide on what you want to do, then look into your ability to legally do this in the property you are considering purchasing.
Water supply:
Check out what sort of water supply the property has. Can you drink water straight from the tap? Is it set up with a tank (and if so, how large is it)?
Consider local amenities:
If you have children (or are planning on having children) you'll want to research the local childcare facilities and schools. How far away they are, school size and quality are all important factors to consider. Make sure the property you are considering is in zone for the school of your choice.
You may also want to look at the distance to the closest doctor, after-hours clinic, and hospital. If there is a medical emergency, you may want to reconsider if the closest medical attention is an hour or more away.
Land topography
Check distances from local rivers and wetlands to ensure that the property is not a flood risk. How hilly the property is may also be worth looking into, as if there is a flood risk you'll want to move any stock up.
Also consider any native trees on the property, and look into any legal regulations in removing them if you are considering building or doing any landscaping work. If you are considering using the soil for growing, soil type will be important. If you can, take a spade to the property and dig a small hole in the land to check the topsoil depth and soil quality.
Boundary lines
Boundary lines are often argued over in rural areas - it can be difficult to determine who owns what land and who is responsible for looking after it, especially if the area is unfenced. If you are looking at fencing the property, you may need to have a surveyor look at your boundary lines.
Check what comes with the property
Make sure that any amenities (eg a septic tank, trough or electric fence) come with the property when you buy it and make sure they are in good working order.
Broadband and cell phone reception
If you are reading this online, chances are the Internet will be important to you. Check cellphone and 3g/4g reception, and check with local internet providers to see if a new broadband line will be able to be set up (don't just assume that if the neighbour has the internet you'll be able to get it as this is not always the case).