Companies trying to break the grip on rare-earth magnets—a crucial ingredient in modern technology—are encountering unusual hurdles.