It all started with "Mary had a little lamb...". Those were the very first words ever recorded, and they were said by Thomas Edison into the horn of his new and amazing sound recording device, the phonograph. That was on December 6, 1877, in the town of West Orange, New Jersey, and that first recording was carved by a needle into a layer of tinfoil (yes, tin; not aluminum) wrapped around a revolving cylinder.
Frankly, the sound was awful scratchy and barely recognizable, but that a recording had been made at all was phenomenal and ushered in a whole new era of technology and, soon, entertainment.
It's not that there had been no way to record music before: When you stop to think about it, this started when the first composer wrote the first musical score for himself or someone else to play at some future time. And it was continued with the invention of the "player piano" which, from pre-punched paper rolls, was able to duplicate each of the notes, but not, originally, the dynamics, of the music. That was reserved for the later Vorsetzer ,  which was able using specially-prepared sheets to record and play back not just the music but the dynamic and emotional content of the original performance, as well.
Even the Vorsetzer, though, was just recording a template for the playing of the music; not its actual sound. That was Edison's accomplishment, and was the ultimate source of our Hi-Fi hobby.
Even so, Edison near-completely ignored the phonograph for the first few years after its invention. He did eventually come to see, however, that there was a market for music to be played in the home or elsewhere and, in 1888, little more than a decade after the first recorded sound, he formed the Edison Phonograph Company to make and sell the first commercial recordings. Those were still cylinders and were still recorded vertically, with the groove dug more or less deeply into the surface in accordance with the changing pressure waves of the sound.
Although the original for-sale recordings were mechanically reproduced, just a few units at a time, changes in materials -- from the original tinfoil, to paraffin-treated-cardboard, to (by the end of the 1890s) a hard black wax that was easier to cut, quieter, better sounding, and that could be played hundreds of times before failing -- eventually made them (in combination with an inexpensive wind-up hand-crank player) a truly viable and broadly-available commercial product.
Besides its longevity, the best thing about the black wax material was that it could also be molded with the sound (the recorded groove) already in place and no cutting or other mechanical duplication necessary. That change was made in 1902 by Edison's National Phonograph Company and the cylinders were called "Edison Gold Moulded Records".
The material changed again, in 1908, to "Amberol", a harder wax, and yet again, in 1912, to celluloid, a material Edison had been involved in developing, and upon its adoption, the name was changed to "Blue Amberol". The real change for 1912, though, was not that, but Edison's introduction of the Edison Diamond Disc Record.
These were still vertical (more or less depth to the groove) recordings, but the disc format, by a speed change and by making the grooves closer together, was able to nearly double the total playing time to as much as five minutes, with materially improved sound quality.
These were, as might be imagined, very successful, but because Edison refused to share his patented recording technology with other companies -- sort of like Sony and some of the things (Betamax, for example) that it has developed -- other companies, including Victor, Brunswick, Silvertone, Columbia, Decca, and many more, went to a lateral (side-to-side instead of deeper-and-shallower) recording format, which eventually surpassed Edison's own and eventually became the standard for 78 rpm recordings and continued to be the standard for disc recording (78s, LPs, 45s, 45EPs, and 16 2/3 rpm ULPs) all the way through the era of monophonic sound.
The format wars were on, as early as 1912, and have continued to this day.
The next big change after lateral disc recording came about in 1925, when many of the same companies (Victor, Brunswick, Columbia, etc.) went to electrical instead of acoustic recording. Edison followed in August of 1927, but it was too late and that company failed only a few years later.
Other changes, mostly in disc size (from the standard 10 inch to the 12 inch LP and the 7 inch"45" and 45EP) also came in over the years, as did such recording-time extending processes as "DynaGroove" and other variable pitch (groove spacing) technologies. The biggest change of all, though, came in 1957, when, after stereo sound which had been introduced to the public in the 1940 Disney film Fantasia; was, by the early 1950s, nearly ubiquitous in major movies; and had even enjoyed some popularity with the audiophile community in the form of pre-recorded tapes -- was first introduced as the stereo LP.
In order to make stereo disc recording and playback practical (unlike other, not-so-practical, disc-based systems, like that used by Emory Cook in his Cook "Binaural" Records, which had already been tried and found to have insuperable difficulties) a new and unique disc-cutting system had to be developed to record two distinct stereo channels in just a single groove. One approach that had been tried was to use a single disc-cutting stylus to cut a vertical signal (a la Edison) and a lateral channel (the then-standard) into the same groove at the same time. This actually did work and (although it was suspected that the vertical cut might wear faster or be noisier) was in contention with a competing system that rotated both channels 45 degrees, until the very first commercial stereo LP was cut and offered for sale in November of 1957. This was from Sidney Frey's company, Audio Fidelity Records, and, once the first 45/45 stereo phono cartridge (an Electro-Voice ceramic) was released to play it, the die was cast and 45/45 became the industry standard forever after.
The biggest format change of all, though, came in 1982, with the release of the CD and its claimed "perfect sound forever."
With the advent of digital sound, audiophiles and music lovers everywhere shouted their hosannas and, either sold or copied their LP collections, abandoned their record players, and flocked to the new medium, which, just as lovers of tube electronics quickly found the solid-state gear that had replaced it to not be as good, audiophiles very often found not to be up to LP standards of sound quality.
That conflict not even mentioning the still on-going controversy over tubes versus solid-state -- has continued to this day, and its results are all around us. Digital has continued its inexorable advance; first from the number of bits our DACs deal with, to different recording and playback algorithms, to different scan speeds and bit counts, to a broad expansion of function into every area of our hobby, including not just recording and playback media, but digital crossovers and room equalization and, as digital has advanced, it has diversified, splitting and splitting again into any number of formats and applications.
Within the no-longer-all-that-new digital realm, all kinds of things have happened since the 1980s. These sticking just to recording and playback formats mostly have had to do with playback quality (HDCD, for example, and SACD, and perhaps most recently, MQA) and now there's also the whole new area of digital streaming which, while accommodating other innovations, eliminates physical media entirely (no LPs, no discs either CDs or DVDs and no tape, either pre-recorded or do-it-yourself) so that on-hand (on-site?) discreet storage, of individual recordings can be completely eliminated and replaced by a single server or equivalent.
So, is that the final format?
I don't think so. While that's been going on, and even for years before, whatever has been set aside or seemingly discarded from our hobby at one time seems to have reappeared at another: Tubes were gone and are now back in full strength. The phonograph was gone and now its reappearance and reimplementation is one of our industry's strongest growth areas. And even tape in the form of copies of analog master tapes to be played on resurrected reel-to-reel recorders (and even some exotic and very expensive new ones) is coming back to an audience that may now be even bigger than it was pre-1957.
Is this confusing? Is it a problem? Or is it an embarrassment of riches? Personally, I'm delighted to see more choices becoming available. It means at least two things to me: That our hobby and our industry may be stronger than I had feared, and that there's more stuff for me to choose from when I want to just turn on the System, lean back, close my eyes, and...
Enjoy the music.