\rho is supposed to control whether the kernel matches exclusively subtrees (all the way to the tips) or subset trees. Setting rho=0 should result in a subtree kernel. The resulting scores are failing normalization --- sqrt(k(t1,t1) * k(t2,t2)) is less than k(t1,t1).