The preferred habitats for the vast majority of microorganisms in soil are particle surfaces. Rather than studying suspensions of soil microorganisms, it is therefore desirable for elucidating mechanisms governing soil microbial diversity, to look at surface attached microbial communities. In our investigations we adjusted a soil fractionation technique for isolating soil particles size fractions which still preserve a significant amount of their attached microorganisms, including the domains Bacteria, Archaea and Fungi. Analyses of soil particle fractions demonstrated by TRFLP (terminal restriction fragment length polymorphism) of PCR-amplified SSU rRNA genes from directly extracted soil DNA differently structured microbial communities on coarse sand, sand, silt, and clay fractions. The abundance of the three domains could be determined by real-time PCR of the same target genes. The method was applied to study three agricultural soil variants generated by a more than 100-year lasting long-term fertilization practice, including no fertilization (NIL), mineral fertilization (NPK) and mineral and organic fertilizations (FYM), resulting in differently sized pools of soil organic carbon and total microbial biomass. Soil variants with higher organic carbon were characterized by higher SSU rRNA gene copy numbers for all three domains. The largest community sizes were detected for the clay and coarse fractions. The bacterial community responded in its size and diversity to soil organic carbon in the larger but not in the smaller soil particle size fractions. We also analyzed the capacity of soil microorganisms to degrade the two organic pollutants phenol and 2,4-Dichlorophenol (2,4-DCP) in the different soil variants utilizing stable isotope probing (SIP). While soil organic matter, as revealed by comparison of the soil variants, affected quantitatively the diversity of the phenol degrading bacteria, it did not affect the fungi. In contrast, 2,4-DCP was only degraded in the fertilized soil variants and bacteria utilizing the compound for growth were different in NPK and FYM. In summary our study demonstrates that soil fractionation can give access to ecologically meaningful, differently structured microbial communities and thereby enhance our capacity to reveal the factors that sustain and modify soil microbial diversity.