Audio can be stored in either an uncompressed format, or a compressed format.  Unlike photos though, these do not always signify that it is a lossless or lossy recording.  For example, an audio recording could be stored in an uncompressed format, but its sample frequency could be lowered to get into that format, making it lossy.  Likewise, there are compressed formats that are lossless, where they make the size of the data smaller, while still retaining all the information about the audio.  Finally, there are compressed and lossy formats, these are the most common for use among consumers of audio.

Uncompressed Formats

Digital audio has a few, very common uncompressed formats that it can use.  They take PCM audio with the parameters described above, and wrap it up into a neat file.  The two main formats you will find are CD Audio and WAV. You will often find other formats that simply specify a PCM type (linear, mu-Law, A-Law), a size (8-bit, 16-bit, 24-bit, etc), signed or unsigned, a frequency (32kHz, 44.1kHz, 48kHz), endianess (big or little), number of channels, and bit-rate. 

CD Audio - The most common format you will find is CD audio, it is uncompressed audio.  This format is 16-bit, Linear Pulse Code Modulation (L-PCM), signed data, running at a 44.1kHz sampling rate, with two channels of audio (stereo).  It runs at 1411kbps 

WAV - Unlike CD Audio, that specifies exact parameters, WAV files are capable of containing many different formats, however the most common one that it contains is identical to the CD audio format: 16-bit, L-PCM, signed, 44.1kHz.  WAV files are almost always found with uncompressed data, however it is possible to use compression inside them.

AUP - This is the format that is used internally by audacity.  When you save a project in audacity, it creates a file with a .aup extension as well as a directory with the same name which stores all your raw audio.  By default this is 44.1kHz 32-bit float data, but that can be changed in the preferences.

Compressed Formats

Digital audio can take up quite a bit of space.  Lets look at typical CD audio: 16-bit samples multiplied by 44.1kHz multiplied by two channels gives 1,411,200 bits/second.  This is equal to 176,400 Bytes/second or 176KB/s or 10.6MB/min or 635MB/hr.  These numbers will look nice and tame when we get to the raw video rates in the next section, but they are still a lot to store, especially when a lot of that space is basically redundant.  There are a few compression formats that use lossless compression, so that no information is lost.  The leading one of these is called FLAC, but there are also others from Apple and Microsoft.  When using a lossless compression format, you can typically compress the audio down to about half of the original size.  However, the majority of audio formats are lossy formats.  The are used for everything from the audio on your DVD and BluRay disk, to MP3 players like the iPod and cellphones, and even podcasts and skype.  When using some of the better lossy compression formats, you can usually re-produce the audio so that it sounds almost excatly the same, using only about 10-20% of the original size.

MP2 - Stands for MPEG-1 audio, layer 2.  Often used in broadcasting.

MP3 - Stands for MPEG-1 audio, layer 3.  This is the most popular audio format, as is evidenced by the number of MP3 players available at electronics stores.  Almost all computers and consumer audio players can play this format.  However, this format is older and Vorbis, AAC, and WMA all offer superior compression ratios for the same quality audio.

AC3 - This is also known as Dolby Digital sound, made popular by its use in movie theaters.  It is used on many DVDs and supports 5.1 channel sound. 

FLAC - Free Lossless Audio Codec.  This audio format was made to take uncompressed formats and shrink them down, without losing any quality at all.  Audio in this format can be converted back to an uncompressed format, and the resulting file would be identical to the original uncompressed file.  This is often used for editing and archival, where you want to be sure not to lose any information.  Files are usually about half the size of an uncompressed file.  This format is open source, and free of royalties, so it can be used anywhere.

Ogg/Vorbis - This is the leading open source compressed audio format.  Its compression is approximately twice as good as MP3, and on par with AAC or WMA compression.  This is the official standard for audio used in web pages.

AAC - Advanced Audio Codec  This is Apple's primary audio codec. It is available on Macintosh, in iTunes, and on iPods/iPhones. 

WMA - Windows Media Audio.  This is Microsoft's primary audio codec.  It is available on Windows computers and many portable devices.

Speex - This is a codec that was developed specifically for compressing human voices.  It is not a high fidelity codec, and should not be used with music or sound effects, but is good at accurately re-producing the human voice.