My family was poor growing up and I was also very poor when I was young and a young mother many years ago. 'Food deserts' were not a factor in our lives. Money was the biggest factor. My parents either didn't have time or energy to make breakfast because they worked long hours for low pay, so I grew up on cheap off-brand sugary cereals. As a kid, I thought it was great. I didn't know how unhealthy that was. Pasta, potatoes, white bread and ground beef were staples and the only vegetables we usually got were canned corn and green beans. Maybe fresh carrots. Sometimes we'd get canned fruit (in syrup, of course). Canned food doesn't go bad quickly and so doesn't get wasted; you can't afford waste when you are poor.
When I became a mother, fresh foods were available, I simply couldn't afford them. Every dollar mattered when I went to the grocery store; fruit and vegetables were luxuries to me.
Time was also an issue. I worked a lot and was too tired to learn to make anything other than simple meals, if I even had time to make meals (and there was a short time period when I often had to rely on the McDonalds dollar menu because it was all I could afford or had time for between working two jobs and having a kid as a single mother). I did what I could to keep things healthy, but the cheap staples tend to be simple carbohydrates. I wouldn't get sugary cereals, but the effort to make even a simple, good breakfast can be overwhelming when you are working two jobs and barely have time to think.
School lunches were also an uphill battle; they weren't healthy and, when I could, I made lunches. It was hard to compete with the stuff the schools served up, even though I tried to teach my kid about healthy eating. Plus, at my poorest moments, I couldn't help but take advantage of the free lunch programs.
I can definitely agree that stress would be a factor in terms of health when one is poor. I was constantly worried about keeping up with rent, food, how am I going to pay for daycare so I can go to work to pay for daycare, etc. Plus, going to the doctor was out except in the case of dire emergencies. No dentist ever. I went to the dentist once as a kid and not again until I finally got health insurance in my late 20's.
As for tackling obesity, well, unhealthy, fattening foods are easy, cheap, widely available and taste great.
It's not a matter of making fresh foods more available, but rather making them more affordable and more desirable. A real move to healthy school lunches. An educational campaign on portion control and caloric intake. Making fried and other unhealthy foods less desirable. Not allowing sugar cereals to be constantly advertised as 'part of a balanced breakfast' (another uphill battle I had as a parent). Healthier easy breakfast options widely known and available. Decreasing soda intake. These are a few places to start, I think.