Ah, Durge ... A bit simplified, but ...
With TOSLINK, no sound output processing is undertaken by the Realtek. Bits are moved from the computer to the receiver via PCM (lossless and not compressed). ALL processing (upmixing, decoding, digital to analog conversion, amplification) is done by the receiver. Bits are bits (data). Generally, even a modestly priced receiver will have vastly superior op-amps, DACs, isolation, filtering than that of any dedicated sound card which has to work in a very noisy electrical environment.
That said, when the Dolby Digital Live (DDL) codec is introduced, then bits are modified in real-time before being pushed out the cable. Technically, DDL degrades sound quality - 48 kHz/16-bit @ 640 kbit/s lossy and compressed. DTS is the DDL competitor. The DTS Interactive codec also ouputs lossy, compressed 5.1 albeit usually 48 kHz/24bit. As a side-note, the sampling frequency of audio CDs is 44.1 kHz and DVDs is 48 kHz. Do not confuse Dolby and DTS used in commercial environments (e.g. theaters) with that of the PC based codecs.
On a PC, the limitation is the TOSLINK connection itself. You can't push more bits through it than it's capable of handling. As an analogy, you can't magically transform SATA II to SATA III through software or otherwise.
If using headphones through the jack located on a PC, then of course, a good dedicated sound card will likely perform better than any on-board chipset. If using TOSLINK, the headphones should be plugged into the receiver to take advantage of the hardware located within. Likewise, if using the analog jacks on the back of a PC, a dedicated card will produce better sound.
When talking analog, higher end mobos generally have better filtering, shielding and isolation. Some boards allow swapping of op-amps. This stuff costs money, so don't expect much from H series Giga boards for example - you'll likely do better with a dedicated card.
On paper, the specs for the Realtek 1150 are fairly impressive. It supports 16/20/24-bit SPDIF, 44.1k/48k/96k/192kHz sample rates and 115dB S/N. However, how it's implemented by the mobo manufacturer and the quality of the Realtek DACs, filters and amps can't match that of a good dedicated card. But with TOSLINK, much of that is irrelevant since none of it is used.
The DTT3500 is no match against a Home Theater system using TOSLINK off the Realtek. By it's own spec sheet, the DTT3500 only outputs 89w RMS total and with 10% THD across all speakers to boot. Also ~70dB S/N. And, the tiny 30w LFE sub can't compare to a floor shaking, wall cracking 150w 15" Polk Audio LFE. Sorry, not even close.
See for yourself:
www.anandtech.com/show/677
By the way, I'm a sound purist who dislikes digital sound (though it is getting much better). Old vinyl through a dedicated, high-end pre-amp and amplifier still sounds better to me. Apparently, a lot of people agree judging by the surge of vinyl sales as of late - could be just a fad though.