The seat for a stone can be cut into prongs by a variety of applications. Although the most common use is a setting bur or hart bur for a round setting, placing an oval stone into a four prong head or crown frequently eliminates this possibility. In this scenario the individual seats are cut into each prong with a combination of hand tools; generally saw frame, files, and gravers.
Although production stone setting often relies on use of a flexible shaft or pendulum drill and rotary cutting tools, the most reliable and controlled method is removing the metal by hand with simple tools. This approach provides more precise control of the outcome as well as allowing less opportunity for damage to the setting or the stone, or for injury to the setter from a momentary loss of control of a rapidly spinning tool.
There was an earlier comment on this thread about prongs being used for faceted stones and bezels being used for cabochon stones. This could be the perception of limited exposure to jewelry, or contact with examples which are primarily commercial jewelry or that which relies upon assembly of commercially produced findings. In the larger realm of jewelry design there is no such categorical limitation in the use of settings. Many jewelry artists fabricate prong style settings for cabochons and set faceted stones in bezels. To the skilled jewelry maker/designer/goldsmith any combination is possible.
Stone setting is a very specialized part of jewelry making. One can endeavor to teach one's self but in my experience they will very likely adopt bad habits and develop incorrect approach in addition to investing much time and energy with poor results. The best way to learn stone setting is by taking a workshop or classes with an expert. The money invested is well worthwhile if this is something which will be a major part of one's body of work and will necessitate proficiency and reliability.