Potential energy exists when a force acts upon an object that tends to restore it to a lower energy configuration. This force is often called a restoring force. For example, when a spring is stretched to the left, it exerts a force to the right so as to return to its original, unstretched position. Similarly, when a mass is lifted up, the force of gravity will act so as to bring it back down. The action of stretching the spring or lifting the mass requires energy to perform. The energy that went into lifting up the mass is stored in its position in the gravitational field, while similarly, the energy it took to stretch the spring is stored in the metal. According to the law of conservation of energy, energy cannot be created or destroyed; hence this energy cannot disappear. Instead, it is stored as potential energy.
No, potential energy is not only because of gravitational force as mentioned above in the example of spring as well.
And, potential energy is because of state of the object with respect to its lower configuration.