Do not think, when Hashem drives away the Canaanites from before you, "Because I am righteous Hashem brought me here to inherit this land and because the original residents were wicked Hashem disinherits them." No, it is not that you are righteous, it is because they are wicked, and because Hashem promised the land to your forefathers.
The Shem Mishmuel brings a Medrash Rabba Breishis 50:11 to explain what error they were being warned about.
It might be that Chazal are telling us that the idea is totally false, that what the Isha Hatzarfis and Lot thought, and what Bnei Yisrael might have thought, was simply wrong. But I do not believe this is true. We can make mistakes on our own, and if Chazal tell us what some biblical figures were thinking, unless they tell us clearly that they are talking about something which is false, it should be taken seriously.
But I don't believe that this explains our Medrash. Lot is one thing: the Isha Hatzarfis is something else entirely. Everything we know about her tells us that she was a holy and righteous person. Hashem sent Eliahu to find shelter in her home, and she merited an awe-inspiring miracle. We have absolutely no right to say that she willfully refused to allow Eliahu's presence to inspire her or elevate her spirituality. On the contrary. The operative principal ought to be טוב לצדיק טוב לשכנו.
1. In the censuses in the Midbar, Shevet Levi was the smallest of the tribes. One of the reasons is that their work in the Midbar involved a constant presence in the Mishkan and the carrying of the holy utensils (see, e.g., Rashi in Vayeitzei, Breishis 29:35.) If a Levi was carrying the Aron Kodesh, and his mind wandered, and for a moment he thought about something trivial or foolish, he would die on the spot. Similarly, a Kohen Gadol, while in the Kodesh Kadashim, had absolutely no leeway. Any infinitesimal failure or distraction would be immediately fatal. Being in the area of such holiness creates a condition of Middas Hadin. The same may be true of the environment of a Tzadik, who embodies the kedusha of the Beis Hamikdash.
2. Similarly, we have a passuk in our parsha that says that the degree of Hashgacha on Eretz Yisrael is qualitatively different than that of the Hashgacha on the rest of the world. Devarim 11:11-12 והארץ אשר אתם עברים שמה לרשתה ארץ הרים ובקעת למטר השמים תשתה מים. ארץ אשר ה' אלהיך דרש אתה תמיד עיני ה' אלקיך בה מרשית השנה ועד אחרית שנה. If so, we might say that the degree of Hashgach in the proximity of a tzadik is far greater. Greater Hashgacha means close scrutiny, the kind of examination that every person eventually undergoes, but usually only at the judgment of Beis Din shel Maalah in the Olam Ha'emes after death. Behavior that would otherwise pass might not survive this kind of close scrutiny.
I personally don't like this pshat. To say that being near a tzadik is like walking under a קיר נטוי to the extent that it operates to turn upside down the whole concept of טוב לצדיק טוב לשכנו, and that there is some chiluk between them, doesn't appeal to me. If you like it, זאלסט דו זיין געזונט
3. When a tzadik lives near you, you realize that you could be greater, that you should grow. Unfortunately, inertia makes growth difficult. Even if we know we ought to change, it is difficult to act upon what we know. Sometimes, people only change in reaction to a traumatic event that forces them to reexamine their lives. In cases like that, Hashem might help that person take that difficult step by shaking them up, by bringing difficult challenges to them.