The prince made the comments to Tom Bradby during the filming of the ITV documentary around their Africa tour.
Asked if the couple intend to spend more time on the continent, Harry said: "I don't know where we could live in Africa at the moment.
"We've just come from Cape Town, that would be an amazing place for us to be able to base ourselves, of course it would.
"But with all the problems that are going on there I just don't see how we would be able to make as much difference as we want to without the issues and the judgement of how we would be with those surroundings."
He continued: "I think it's a very hard place to live when you know what's going on but then you're sort of slightly disconnected from it. So, the rest of our lives, our life's work will be predominatley focused on Africa, on conservation.
"There are 19 Commonwealth countries across this continent so there's a lot of things to be done, but there's also huge potential."
ITV News explains that Harry and Meghan will be taking some time away from their Royal duties in the upcoming months - some of which they will spend in the US - as they feel that they are "existing not living". He also questions what the couple's role may be in the future
Harry & Meghan: An African Journey airs on Sunday at 9pm on ITV