I used XBMC when I had a computer hooked directly up to my TV (an NVidia Mac Mini) and it worked fine. I had four external HDDs hooked up and it was becoming untenable, plus left me with a single TV that had access to my files.
At the time there was limited multi-client support in XBMC and it was hairy to say the least.
I moved to Plex, which at one time was an XBMC spinoff, and offered single-system transcoding for all of my clients, and offered non-transcoded files for my Mac mini running the Plex client software. It also offered the ability to stream to my mobile clients, anywhere in the world at any time. I'd fire up my Android tablet, or later my iPad or any of my phones (except for the Windows mobile one...) or a web browser and could work with my library, use any of the remote channels, or watch any of my media stored on the server. After using it for about a year on the Mac Mini, I moved the server side to a dedicated box I built with a lot of storage, as I'd rip my BluRays and HD-DVDs in raw format MKVs and wouldn't encode them at all. It was a basic i3 box with 8GB of RAM and cost about $350 to build, outside of the hard drives (12 4TB plus JBOD SATA cards; same as used in Backblaze StoragePods).
Plex is good in many respects but clearly lags in Live TV integration (have a SiliconDust Connect and a SiliconDust Prime; only use Connect now), so I've actually been working on migrating to Emby. Has the great metadata management that both XBMC and Plex have, multi-client use, and the Live TV integration built-in and remote client streaming.
I have native Plex clients still running on that Mac Mini (no hiccups over wired Ethernet to computer, running full 1080p, uncompressed 35mbit on Avatar BluRay MKV rip) and have DLNA access through a PS3 and native client on Roku (decent enough for the bedroom TV).
For a first timer, I'd setup Plex on the computer that you have your files on and use a Roku 3. There may be some instances where it's not the best picture, but for 99% of files, if you have a halfway decent wireless network or the ability to wire in Ethernet. If it ends up working, it's easy to procure a small PC, like a NUC or a Mac Mini that can run the Plex client and run full uncompressed MKVs from BluRays without any hiccups. I will note that with my Roku 3, I've only had hiccups very rarely and only on the most demanding files. I've got a set of HD test videos that test VC-1, H.264 and H.265 codecs on devices and the Rokus have fared well.
One thing I will note; an MP4 and an MKV are not entirely comparable file types. An MP4 file is typically an MPEG-4 encoded video/audio file. An MKV is a container file format that can contain encoded streams of different formats. Your MKV can and often has an MPEG-4 encoded video stream embedded within it. BluRays can have MPEG-2, MPEG-4 (h.264) and AVC VC-1 codecs used for videos, and my ripping software (MakeMKV) simply dumps the decrypted stream into an MKV container at whatever format it was in. MP4 doesn't necessarily mean a lower-quality file, but rather it's dependent on the encoder's quality settings. One thing about MP4 is that virtually every media playing device available these days can decode an MP4 file and play it. Certain other formats aren't as ubiquitous.