We knew this going in, and it turned out to be true - the steeper the land you're working with, the more complicated the whole process will be. That meant more money spent on the driveway, less flexibility with respect to actually choosing a building site on the property, and importantly with respect to prefab, more difficulty in actually getting the module up to the site (with the attendant stress on the module: drywall cracking, flex on the windows, anxiety for me when the box actually got stuck on the tight turn). On the flip side, the steepness of the site means we get a view. So we've got that going for us.
Second, and this was hugely important, I think, the decision to go with unrestricted land as opposed to buying in a sub-division was the right one for us. It means we don't have to worry about aggravating a homeowners' association when build a modern home or rent it to weekenders. It meant some more money up front for the driveway and utility infrastructure, but we probably got a bit of break in $/acre in buying unimproved land in the first place.
West Virginia - I have to say I was a little nervous about picking a site in WV. Wasn't sure about the mix of people, or the proximity to services, or how receptive guests would be to the idea of a high-end cabin in WV. But all in all I think things have been great. The people have been wonderful, from all the guys working on the house to our neighbors and the businesses in the area, just very welcoming of new people. (Part of that, I think, is a function of being so close to a town that is so popular with DC's gay community.) The light regulation of development (and the absence of any building code or inspection process) meant that we could move the process along much more quickly than we could have in VA or MD, where just getting a building permit could take a year. On the down side, and speaking generally, the local contractors we used did not have a familiarity with a lot of the modern style materials that we spec'd. We did a lot ourselves, so it wasn't an issue, but I have to say I would be really nervous trying to build an affordable modern house with builders in a really rural area that haven't done it before.