Objective To investigate the relationship between diversity of farmland small rodents and environmental factors in different regions of Shanxi province, China and to provide a foundation for regional integrated management for farmland small rodents in Shanxi province. Methods The study areas were established in Xi county of Linfen, Loufan county of Taiyuan, and Wutai county of Xinzhou in Shanxi province. The snap?trap method was used to investigate the diversity of farmland small rodents in each study area from March to November. The vegetation heterogeneity index in the middle ten days of July was used to determine the habitat complexity in study area. The impact of farmland habitat complexity on diversity of small rodents in Shanxi province was evaluated. Results There was a significant positive correlation between the richness and diversity indices of farmland small rodents and vegetation heterogeneity index in Shanxi province, with a correlation coefficient of 0.998. On the other hand, the species dominance index was negatively correlated with the vegetation heterogeneity index, with a correlation coefficient of -1.000. However, the capture rate and evenness index showed no significant correlation with the vegetation heterogeneity index, with correlation coefficients of 0.404 and 0.994, respectively. Conclusion The complexity of habitat environment is a key factor influencing the richness and diversity of small rodent community. The uniformity of vegetation coverage in farmland is probably correlated with the evenness index of small rodent community.