Bone marrow mesenchymal stem cells (MSCs) are a valuable cell source for tissue engineering and regenerative medicine. Transforming growth factor beta (TGF-beta) can promote MSC differentiation into either smooth muscle cells (SMCs) or chondrogenic cells. Here we showed that the stiffness of cell adhesion substrates modulated these differential effects. MSCs on soft substrates had less spreading, fewer stress fibers and lower proliferation rate than MSCs on stiff substrates. MSCs on stiff substrates had higher expression of SMC markers alpha-actin and calponin-1; in contrast, MSCs on soft substrates had a higher expression of chondrogenic marker collagen-II and adipogenic marker lipoprotein lipase (LPL). TGF-beta increased SMC marker expression on stiff substrates. However, TGF-beta increased chondrogenic marker expression and suppressed adipogenic marker expression on soft substrates, while adipogenic medium and soft substrates induced adipogenic differentiation effectively. Rho GTPase was involved in the expression of all aforementioned lineage markers, but did not account for the differential effects of substrate stiffness. In addition, soft substrates did not significantly affect Rho activity, but inhibited Rho-induced stress fiber formation and alpha-actin assembly. Further analysis showed that MSCs on soft substrates had weaker cell adhesion, and that the suppression of cell adhesion strength mimicked the effects of soft substrates on the lineage marker expression. These results provide insights of how substrate stiffness differentially regulates stem cell differentiation, and have significant implications for the design of biomaterials with appropriate mechanical property for tissue regeneration.