Thread Number Index
#1 Intro - Media Center Master - PVR Integration - MCEBuddy - 2nd Smaller File [TC]
#6 CAST LISTINGS and TRAILERS https://forums.plex.tv/discussion/comment/793038/#Comment_793038
#7 Personal Media (NFL, Learning Library, etc) - Extended Personal Media Scanner - Extended Personal Media Shows Agent https://forums.plex.tv/discussion/comment/794022/#Comment_794022
#12 Transmogrify for Plex https://forums.plex.tv/discussion/comment/794576/#Comment_794576
#15, #31 Storage Spaces <-- Replaced, see #71
#41 Unofficial Appstore Channel https://forums.plex.tv/discussion/comment/812725/#Comment_812725
#48 Dynamic Libaries <-- See a better way in #131
#59 Auto select library items (duplicate files at lower bitrate/resolution) https://forums.plex.tv/discussion/comment/876423/#Comment_876423
#131 How to create "Virtual Library's from Tags" Useful for "Recommended, Christmas, Season type movie collections" https://forums.plex.tv/discussion/comment/927727/#Comment_927727
#145 MP4 Auto Convert Scripts (remux and transcode) https://forums.plex.tv/discussion/comment/931888/#Comment_931888
SQLite Script to find all MP4 files in your library that are not Web Optimized
https://forums.plex.tv/discussion/comment/1105503/#Comment_1105503
How to setup and use DrivePool and SnapRAID together:
https://forums.plex.tv/discussion/comment/1113655/#Comment_1113655
Introduction
A few people have asked me to start a thread about my system, the setup, the how/why's of it and such. So I decided to do so. Hopefully this will help other people and maybe I can get some additional recommendations from others too. All programs mentioned are for "Windows".
For starters I've been a video junkie for a long, long time. I've had a digital media collection for about 20 years now. I started using BeyondTV (SnapStream) and switched over to SageTV. I had 12 or so tuners recording any and everything I might have been interested in. I'd cut commercials etc. This was state of the art back in the day. Some of old timers might even remember me from both of the forums as I was a big time 3rd party developer/integrator on both platforms.
Jumping forward many years. SD was on the way out and HD was the "new thing". Usenet and Torrents were becoming the norm for aquiring TV shows and Movies. Internet speeds (if you wanted to pay for them) was fast enough that you could download shows faster then you could record and process them. Times were changing. I for the most part went from Recording shows/movies and ripping DVDs in a more or less SageTV environment to having most things these days available via Plex. Tried lots of stuff in between but I'll save you the details.
I love Plex as it makes accessing your media super easy from just about anywhere or any device, but there are "holes" and some "tricks and tips" that make things run easier/better. I also still have a few things I'm presently working to solve or improve on. I've chosen a few item to start to talk about as these issues tend to come up a lot in the forums.
For example while I can have a great collection of media:
1) I can't pull down shows/movies from newsgroups or torrent sites automatically.
2) I can't out of the box do any PVR type stuff (TV Show/Movie recording).
3) I'm not a fan of the way Plex handles music. It also only picks up 2/3 of my music collection.
4) Plex doesn't allow "friends" to download but only sync media (assuming you give them permission) to some devices.
5) I can't stream to more then a half dozen or so people (requiring transcoding) at the same time reliably as the transcoding engine will peg the CPU and cause some buffering issues for remote clients.
So to give you a bit of info about my system. I have 300/300 Internet service via FIOS.
My media:
265- 3D Movies (Bluray Quality and growing quickly)
9,200 - Movies
1,271 - Educations
21,400 - Music Videos
1200 - NFL Games
30,735 - TV Episodes (360 Shows, 1,850 Seasons)
Music: 8,000 Artists, 27,900 Albums, 215,560 Tracks
Edit: updated stats on 5/8/17
Most of my collection is HD (720p to 1080p). Everything is fully indexed and all videos are encoded in h.264 MP4 containers with a stereo track added as the first track (Dolby/DTS tracks are additional tracks). I've processed everything through HandBrake or XMedia depending on source.
My server is shared with about 20 accounts/friends that actively use it. A few more accounts that occasionally use it. Some accounts have multiple Rokus and I'll sometimes I see 3 or 4 active streams from these "one" account/friends. Many of the shared players (ie Rokus) are set at 4Mbps 720p resolution which causes lots of transcodes on my 720p/1080p movies. This is probably due to the Rokus being used on WIFI at the remote location which I can't easily control.
To give some insite and to address the 5 items listed above:
1) I can't pull down shows/movies from newsgroups or torrent sites automatically.
Well this SHOULD NOT be included in Plex for obvious reasons. It is however something that can be setup. There are lots of solutions to this in the forums using tools such as integrating SABnzbd, Sickbeard, Couchpotato and Headphones. This can/will work but it's a lot of setup work and maintenance. I used to do this also but have recently moved away from this jungle of programs.
What I'll suggest instead is to check out Media Center Master http://www.mediacentermaster.com. It will do 95% of what the above jungle of programs will do for you. It DOES take a bit of time to figure out and get configured properly but it's very powerful and just works.
The video below will give you a quick overview of the setup.
Check out a few videos for acquiring TV Shows and Movies on Youtube and then search the forums here for more info on MCM. Worth the switch or initial setup.
2) I can't out of the box do any PVR type stuff (TV Show/Movie recording).
Yep, hard to believe but Plex STILL doesn't have any PVR functionality built in. I've tried numerious PVR application for integration into Plex including MythTV, NextPVR, Windows Media Center, Media Portal. While I was able to get all of them working with Plex I decided I wanted to keep the solution as simple as possible so there would be less to break down the road. Many of the solutions require a lot of hand holding to get EPG working and is tough at times to integrate especially in the USA. I eliminated SageTV and BeyondTV (even though I have them and know how to use them extensively) as neither program is available anymore.
Here are/were my goals for PVR functionality:
A - I only care about the final media file ending up in Plex in a Movie or TV Show folder. I really don't care about needing to view the file while it's being recorded (via plex).
B - I didn't want to use a plug in for plex as they are only available to the owner of the system and not friends or family accounts.
C - I wanted the ability to be able to remotely setup/modify recording via an EPG guide.
D - I wanted the ability to be able to view what is being recorded remotely (aka Slingbox style) and to be able to set the resolution depending on the device I'm using (real-time transcoding).
E - I wanted the ability to be able to remove commercials from some (not all) recordings before archiving to Plex.
F - I wanted the file to end up in MP4 format like my other library items and with a proper file name that Plex understands for meta data purposes.
Not to much to ask for is it?
How I accomplished this. First I purchased a HDHomeRun Prime from NewEgg for $79 (current sale). This device uses a CableCard and has 3 tuners built in. I setup a 2nd computer running Windows 8.1 Media Center Edition ($10 upgrade for windows 8/8.1). MCE just plain worked better then MythTV, Media Portal, NextPVR, etc. The setup was straight forward and it has integraded 14 day EPG guide built in. It does all the basics I needed. Besides setting up the tuners in MCE I basically only setup the recording directories and everything else was at defaults (doesn't get easier than this).
Next I downloaded and installed Remote Potato from http://www.remotepotato.com which allows you to remotely schedule recordings and allows you to play back anything recorded or watch live TV in different resolution/profiles. This one app integrated first shot with MCE and just works. Remote Potato solved C & D above.
Next up was requirements E & F. To do this I installed MCEBuddy https://mcebuddy2x.codeplex.com/. They have a "free" version and a more advanced version that is faster and has more features. They call this EARLY ACCESS PREMIUM PROGRAM. Try the "free" version and after you get the hang of what it does make a donation of $25 to the dev for life time updates to the current version (what you want). Then do the same for ComSkip $10 to get faster commercial cutting. Both are "cheap" in my mind and you get to try the "normal" versions first. MCEBuddy will for the most part cut commercials, convert to MP4 format (huge saving in file space), rename files with proper naming that Plex understands and can put the files directly into your Plex libraries.
You can of course setup some shows to not go through MCEBuddy processing. Example if you want to record the news on a local channel a couple of times a day and have it available in case something happens and you want to be able to refer back to it. Or maybe you want to watch Who wants to be a Millionair or similar and delete the files after viewing...
3) I'm not a fan of the way Plex handles music. It also only picks up 2/3 of my music collection.
4) Plex doesn't allow "friends" to download media but only sync media (assuminig you give them permission) to some devices.
Personally I'd rather let my daughter download a few movies or albums to her laptop and/or phone then "sync" for her half a day bus ride back to college or from college to home. Or to be able to allow a friend to download a few movies to take on vacation with them, or have available for the plain/flight... Syncing normally degrades quality and puts a hurting on the Plex CPU.
Both of these issues will get taken care of here. To me music in Plex is more of an after thought especially compared to the way it handles TV Shows and Movies. It will do the basics if you have all files in the formats it understands. I installed MadSonic http://www.madsonic.org/ (Subsonic fork) on my main Plex server and setup the library paths the same as Plex. So MadSonic has access to the same media as Plex does.
Madsonic handles music much better as you'll find out. It also allows the owner to setup accounts for users and set all kinds of things like the ability to download albums, tracks or movies/videos. You can download single files as MP4 (example) or select multiple files and have them downloaded as a ZIP file. This is quite a nice way to allow a friend to download a couple of albums, etc. Madsonic allows the operator to set quotes and in many ways is more advanced then Plex but it's no where near as visually appealing (to me). Madsonic has clients available for your Windows, Android and Apple phones as well as a web interface.
5) I can't stream to more then a half dozen or so people (requiring transcoding) at the same time reliably as the transcoding engine will peg the CPU and cause some buffering issues for remote clients.
Especially true if you add media and the "indexer" kicks in while also transcoding for remote clients.
The obvious thing you can do is upgrade to the latest/greatest i7 or XEON CPU but this will only take you so far. This to me is the "achilles heel" in the Plex architecture at present. There is only so much CPU to go around unless you want to purchase a quad ZEON server class machine with 12/24+ cores per CPU. I doubt many Plex users are going to spend $20K+ on something like this.![:)]( ":)")
I've brought this up recently in a couple of other threads. There are a couple of "workable" solutions depending on your needs. For example, at present when ever I add a new movie to my collection that I think others are going to watch I create two files. The first file is the normal high res HandBrake encode (high bitrate) 1080p file. The second file is created at 720p resolution with a bitrate at or under 4Mbit. Once both files are added to Plex it can dynamically choose the best resolution to "direct play" if possible. Sort of like a poor mans adaptive bitrate if you will.
I simple use the "normal" file name with a [TC] in the name ie Abduction (2011) [TC].mp4. I've been using the built in transcoder in "sync" mode to generate these files to the "cloud" but have the cloud blocked via my router so they never get updated. I can then grab them and rename them. The [TC] in the file name is just what I decided to call this files. TC=TransCoded. I could have just as easily used [720-4] which would be more descriptive especially if I wanted to do multiple bit-rate files.
This "solution" is of course a trade off of storage space vs CPU use. I'll probably develop this into more of a automated fashion using some logic. For example maybe I would only want to have the most recent 100 movies encoded this way OR give it 1 TB disk space. It would essentially work as a "cache" for the last X movies added that are over a certain profile (for me 4Mbit and/or 720p). This way for example if you were to just add Avatar or Guardians of the Galaxy and knew you'd have friends who would be watching it you'd have them covered.
Another solution to an over-whelmed CPU Plex system would be to clone the setup of the plex server onto another machine. Then manually seperate users to different machines. The downside to this is that anytime new media is added both machines now will have to create index files and pull down meta data etc. Probably not a big deal and this does give you a "backup" machine.
However, a much better solution would be to have the ability to fire up transcoders on multiple machines running on the local LAN. This would be the most ideal use of machines and the transcoders could be used for multiple purposes such as replacing Handbrake for the initial encode of RIPs, encoding PVR material, encoding for on the fly video sent to clients, etc....
MOVING FORWARD
I'm still fine tuning PVR functionality but once past this I'm going to try and tackle some type of distributed encoding/transcoding of Plex so that it can have other computers on the network doing some of the real-time encoding. If done correctly this would help Plex scale out much better for those with an extra computer available. I'm also going to work/play with hardware assisted encoding which is now fully working in some freeware programs such as HandBrake. I don't know if I'd use the hardware assist for long term storage of my BluRays but for the [TC] "cache" files or for quick encoding of PVR material this could be ideal. I'm not sure it wouldn't be a bad idea for the real-time encodes sent to clients but that might be a lot harder to implement.
So that's it for now,
Carlo
PS Convert file script can be found on page 52 for download.
#1 Intro - Media Center Master - PVR Integration - MCEBuddy - 2nd Smaller File [TC]
#6 CAST LISTINGS and TRAILERS https://forums.plex.tv/discussion/comment/793038/#Comment_793038
#7 Personal Media (NFL, Learning Library, etc) - Extended Personal Media Scanner - Extended Personal Media Shows Agent https://forums.plex.tv/discussion/comment/794022/#Comment_794022
#12 Transmogrify for Plex https://forums.plex.tv/discussion/comment/794576/#Comment_794576
#15, #31 Storage Spaces <-- Replaced, see #71
#41 Unofficial Appstore Channel https://forums.plex.tv/discussion/comment/812725/#Comment_812725
#48 Dynamic Libaries <-- See a better way in #131
#59 Auto select library items (duplicate files at lower bitrate/resolution) https://forums.plex.tv/discussion/comment/876423/#Comment_876423
#131 How to create "Virtual Library's from Tags" Useful for "Recommended, Christmas, Season type movie collections" https://forums.plex.tv/discussion/comment/927727/#Comment_927727
#145 MP4 Auto Convert Scripts (remux and transcode) https://forums.plex.tv/discussion/comment/931888/#Comment_931888
SQLite Script to find all MP4 files in your library that are not Web Optimized
https://forums.plex.tv/discussion/comment/1105503/#Comment_1105503
How to setup and use DrivePool and SnapRAID together:
https://forums.plex.tv/discussion/comment/1113655/#Comment_1113655
Introduction
A few people have asked me to start a thread about my system, the setup, the how/why's of it and such. So I decided to do so. Hopefully this will help other people and maybe I can get some additional recommendations from others too. All programs mentioned are for "Windows".
For starters I've been a video junkie for a long, long time. I've had a digital media collection for about 20 years now. I started using BeyondTV (SnapStream) and switched over to SageTV. I had 12 or so tuners recording any and everything I might have been interested in. I'd cut commercials etc. This was state of the art back in the day. Some of old timers might even remember me from both of the forums as I was a big time 3rd party developer/integrator on both platforms.
Jumping forward many years. SD was on the way out and HD was the "new thing". Usenet and Torrents were becoming the norm for aquiring TV shows and Movies. Internet speeds (if you wanted to pay for them) was fast enough that you could download shows faster then you could record and process them. Times were changing. I for the most part went from Recording shows/movies and ripping DVDs in a more or less SageTV environment to having most things these days available via Plex. Tried lots of stuff in between but I'll save you the details.
I love Plex as it makes accessing your media super easy from just about anywhere or any device, but there are "holes" and some "tricks and tips" that make things run easier/better. I also still have a few things I'm presently working to solve or improve on. I've chosen a few item to start to talk about as these issues tend to come up a lot in the forums.
For example while I can have a great collection of media:
1) I can't pull down shows/movies from newsgroups or torrent sites automatically.
2) I can't out of the box do any PVR type stuff (TV Show/Movie recording).
3) I'm not a fan of the way Plex handles music. It also only picks up 2/3 of my music collection.
4) Plex doesn't allow "friends" to download but only sync media (assuming you give them permission) to some devices.
5) I can't stream to more then a half dozen or so people (requiring transcoding) at the same time reliably as the transcoding engine will peg the CPU and cause some buffering issues for remote clients.
So to give you a bit of info about my system. I have 300/300 Internet service via FIOS.
My media:
265- 3D Movies (Bluray Quality and growing quickly)
9,200 - Movies
1,271 - Educations
21,400 - Music Videos
1200 - NFL Games
30,735 - TV Episodes (360 Shows, 1,850 Seasons)
Music: 8,000 Artists, 27,900 Albums, 215,560 Tracks
Edit: updated stats on 5/8/17
Most of my collection is HD (720p to 1080p). Everything is fully indexed and all videos are encoded in h.264 MP4 containers with a stereo track added as the first track (Dolby/DTS tracks are additional tracks). I've processed everything through HandBrake or XMedia depending on source.
My server is shared with about 20 accounts/friends that actively use it. A few more accounts that occasionally use it. Some accounts have multiple Rokus and I'll sometimes I see 3 or 4 active streams from these "one" account/friends. Many of the shared players (ie Rokus) are set at 4Mbps 720p resolution which causes lots of transcodes on my 720p/1080p movies. This is probably due to the Rokus being used on WIFI at the remote location which I can't easily control.
To give some insite and to address the 5 items listed above:
1) I can't pull down shows/movies from newsgroups or torrent sites automatically.
Well this SHOULD NOT be included in Plex for obvious reasons. It is however something that can be setup. There are lots of solutions to this in the forums using tools such as integrating SABnzbd, Sickbeard, Couchpotato and Headphones. This can/will work but it's a lot of setup work and maintenance. I used to do this also but have recently moved away from this jungle of programs.
What I'll suggest instead is to check out Media Center Master http://www.mediacentermaster.com. It will do 95% of what the above jungle of programs will do for you. It DOES take a bit of time to figure out and get configured properly but it's very powerful and just works.
The video below will give you a quick overview of the setup.
Check out a few videos for acquiring TV Shows and Movies on Youtube and then search the forums here for more info on MCM. Worth the switch or initial setup.
2) I can't out of the box do any PVR type stuff (TV Show/Movie recording).
Yep, hard to believe but Plex STILL doesn't have any PVR functionality built in. I've tried numerious PVR application for integration into Plex including MythTV, NextPVR, Windows Media Center, Media Portal. While I was able to get all of them working with Plex I decided I wanted to keep the solution as simple as possible so there would be less to break down the road. Many of the solutions require a lot of hand holding to get EPG working and is tough at times to integrate especially in the USA. I eliminated SageTV and BeyondTV (even though I have them and know how to use them extensively) as neither program is available anymore.
Here are/were my goals for PVR functionality:
A - I only care about the final media file ending up in Plex in a Movie or TV Show folder. I really don't care about needing to view the file while it's being recorded (via plex).
B - I didn't want to use a plug in for plex as they are only available to the owner of the system and not friends or family accounts.
C - I wanted the ability to be able to remotely setup/modify recording via an EPG guide.
D - I wanted the ability to be able to view what is being recorded remotely (aka Slingbox style) and to be able to set the resolution depending on the device I'm using (real-time transcoding).
E - I wanted the ability to be able to remove commercials from some (not all) recordings before archiving to Plex.
F - I wanted the file to end up in MP4 format like my other library items and with a proper file name that Plex understands for meta data purposes.
Not to much to ask for is it?
How I accomplished this. First I purchased a HDHomeRun Prime from NewEgg for $79 (current sale). This device uses a CableCard and has 3 tuners built in. I setup a 2nd computer running Windows 8.1 Media Center Edition ($10 upgrade for windows 8/8.1). MCE just plain worked better then MythTV, Media Portal, NextPVR, etc. The setup was straight forward and it has integraded 14 day EPG guide built in. It does all the basics I needed. Besides setting up the tuners in MCE I basically only setup the recording directories and everything else was at defaults (doesn't get easier than this).
Next I downloaded and installed Remote Potato from http://www.remotepotato.com which allows you to remotely schedule recordings and allows you to play back anything recorded or watch live TV in different resolution/profiles. This one app integrated first shot with MCE and just works. Remote Potato solved C & D above.
Next up was requirements E & F. To do this I installed MCEBuddy https://mcebuddy2x.codeplex.com/. They have a "free" version and a more advanced version that is faster and has more features. They call this EARLY ACCESS PREMIUM PROGRAM. Try the "free" version and after you get the hang of what it does make a donation of $25 to the dev for life time updates to the current version (what you want). Then do the same for ComSkip $10 to get faster commercial cutting. Both are "cheap" in my mind and you get to try the "normal" versions first. MCEBuddy will for the most part cut commercials, convert to MP4 format (huge saving in file space), rename files with proper naming that Plex understands and can put the files directly into your Plex libraries.
You can of course setup some shows to not go through MCEBuddy processing. Example if you want to record the news on a local channel a couple of times a day and have it available in case something happens and you want to be able to refer back to it. Or maybe you want to watch Who wants to be a Millionair or similar and delete the files after viewing...
3) I'm not a fan of the way Plex handles music. It also only picks up 2/3 of my music collection.
4) Plex doesn't allow "friends" to download media but only sync media (assuminig you give them permission) to some devices.
Personally I'd rather let my daughter download a few movies or albums to her laptop and/or phone then "sync" for her half a day bus ride back to college or from college to home. Or to be able to allow a friend to download a few movies to take on vacation with them, or have available for the plain/flight... Syncing normally degrades quality and puts a hurting on the Plex CPU.
Both of these issues will get taken care of here. To me music in Plex is more of an after thought especially compared to the way it handles TV Shows and Movies. It will do the basics if you have all files in the formats it understands. I installed MadSonic http://www.madsonic.org/ (Subsonic fork) on my main Plex server and setup the library paths the same as Plex. So MadSonic has access to the same media as Plex does.
Madsonic handles music much better as you'll find out. It also allows the owner to setup accounts for users and set all kinds of things like the ability to download albums, tracks or movies/videos. You can download single files as MP4 (example) or select multiple files and have them downloaded as a ZIP file. This is quite a nice way to allow a friend to download a couple of albums, etc. Madsonic allows the operator to set quotes and in many ways is more advanced then Plex but it's no where near as visually appealing (to me). Madsonic has clients available for your Windows, Android and Apple phones as well as a web interface.
5) I can't stream to more then a half dozen or so people (requiring transcoding) at the same time reliably as the transcoding engine will peg the CPU and cause some buffering issues for remote clients.
Especially true if you add media and the "indexer" kicks in while also transcoding for remote clients.
The obvious thing you can do is upgrade to the latest/greatest i7 or XEON CPU but this will only take you so far. This to me is the "achilles heel" in the Plex architecture at present. There is only so much CPU to go around unless you want to purchase a quad ZEON server class machine with 12/24+ cores per CPU. I doubt many Plex users are going to spend $20K+ on something like this.
I've brought this up recently in a couple of other threads. There are a couple of "workable" solutions depending on your needs. For example, at present when ever I add a new movie to my collection that I think others are going to watch I create two files. The first file is the normal high res HandBrake encode (high bitrate) 1080p file. The second file is created at 720p resolution with a bitrate at or under 4Mbit. Once both files are added to Plex it can dynamically choose the best resolution to "direct play" if possible. Sort of like a poor mans adaptive bitrate if you will.
I simple use the "normal" file name with a [TC] in the name ie Abduction (2011) [TC].mp4. I've been using the built in transcoder in "sync" mode to generate these files to the "cloud" but have the cloud blocked via my router so they never get updated. I can then grab them and rename them. The [TC] in the file name is just what I decided to call this files. TC=TransCoded. I could have just as easily used [720-4] which would be more descriptive especially if I wanted to do multiple bit-rate files.
This "solution" is of course a trade off of storage space vs CPU use. I'll probably develop this into more of a automated fashion using some logic. For example maybe I would only want to have the most recent 100 movies encoded this way OR give it 1 TB disk space. It would essentially work as a "cache" for the last X movies added that are over a certain profile (for me 4Mbit and/or 720p). This way for example if you were to just add Avatar or Guardians of the Galaxy and knew you'd have friends who would be watching it you'd have them covered.
Another solution to an over-whelmed CPU Plex system would be to clone the setup of the plex server onto another machine. Then manually seperate users to different machines. The downside to this is that anytime new media is added both machines now will have to create index files and pull down meta data etc. Probably not a big deal and this does give you a "backup" machine.
However, a much better solution would be to have the ability to fire up transcoders on multiple machines running on the local LAN. This would be the most ideal use of machines and the transcoders could be used for multiple purposes such as replacing Handbrake for the initial encode of RIPs, encoding PVR material, encoding for on the fly video sent to clients, etc....
MOVING FORWARD
I'm still fine tuning PVR functionality but once past this I'm going to try and tackle some type of distributed encoding/transcoding of Plex so that it can have other computers on the network doing some of the real-time encoding. If done correctly this would help Plex scale out much better for those with an extra computer available. I'm also going to work/play with hardware assisted encoding which is now fully working in some freeware programs such as HandBrake. I don't know if I'd use the hardware assist for long term storage of my BluRays but for the [TC] "cache" files or for quick encoding of PVR material this could be ideal. I'm not sure it wouldn't be a bad idea for the real-time encodes sent to clients but that might be a lot harder to implement.
So that's it for now,
Carlo
PS Convert file script can be found on page 52 for download.