The secretary of state’s talks with Xi Jinping raised hopes that the world’s two biggest economies might stabilize relations, but the big challenges were not resolved.