MegaEPG EPG grabber

The MegaEPG is an Electronic Programs Guide grabber. It offers an easy and effective way to collect EPG from the Internet sources and to store the EPG into the popular XMLTV or Windows MCE/Vista native formats. The MegaEPG provides a rich set of features:

No prerequisites, period. It does not require installation of additional platform/run-time environment (no .NET, no Java/JRE, no 3d-party software). It works just out of the box. It does not install anything into the system directories and cleans up completely when removed.
Windows Vista and Media Center Edition 2005 direct import (native guide format). No 3d-party software required.
A comprehensive processing, obtaining as many of XMLTV and MCE attributes as possible: categories, movie length, production year, episode number, original name and so on.
IMDB rating for movies.
Support for alternative listings for the same provider (e.g. ynet, walla in Israel).
Rich scripting API to apply for new listing sources.
Support for automatic alternative listing choice.
"Smart" show descriptions downloading, including weekends and holidays. Official Israel holidays are supported.
Support for different languages (e.g. Hebrew and Russian in Israel) and even mixing languages within the XMLTV.
Support for Unicode and different encodings.
Easy-to-use configuration wizard.
Cache system reduces the amount and time of download dramatically. Re-running the MegaEPG with the data cached might complete in seconds.
Extremely configurable.
Platform independent and ready to serve Linux-based media centers. Works in Linux smoothly with the wine project.
Fair use of providers' resources. Makes delays between channels downloading, yet EPG processing is still very fast.
Support for channel icons.
Categories translator into standard Windows MCE categories.
Log emailing feature that allows you to receive the status of EPG downloading and alert on problems.

The MegaEPG currently supports the following countries: Israel, Norway and South Africa. If you are interested in adding another country or another provider in the existing country, please leave a request on http://pvr.co.il

Download and run the megaepg_setup executable (Linux users should run it under wine). If you are upgrading from a previous MegaEPG version, it will be uninstalled first (existing settings will be preserved).
Select the country you are in or ensure the correct country is chosen. Do NOT continue installation if your country is not listed!
Follow the installation wizard and choose the directory to install. It is not recommended to change the installation directory. The wizard will automatically detect whether Windows MCE operating system is running and suggest the correct directory. Note, the MegaEPG must be installed under the windows\ehome directory in Windows MCE.
Configure the MegaEPG. The wizard will execute the MegaEPG configurator (epgconfig) upon completion by the default.
Schedule the MegaEPG to be executed at least daily.
Run the megaepg.exe (or megaepgw.exe) in order to obtain the initial guide.
Setup your PVR/mediacenter application. Refer to the Windows MCE section or Windows Vista for instructions. SageTV setup instructions are to be written...

The MegaEPG reads the configuration settings from the megaepg.ini file. The configuration wizard epgconfig helps you to configure the most important settings. It is very important to create a correct configuration before the MegaEPG will start bringing program guides for you, therefore configure the MegaEPG right upon the installation is complete. The epgconfig can be used every time you need to change the configuration (advanced users may also edit the ini file manually). If you want to apply a filter script e.g. the IMDB movies rating script, you should do that manually as well. Refer to the filters configuration section for the details.

Choose the system language from the list. The system language will be used for channel names, categories and language attributes of the XMLTV file. Please note, the language does not define the TV guide language (it's defined by the Internet source and the script), the language only applies to channel and categories names. Some countries may support a single language only.
Choose the application type (Windows MCE or a XMLTV compatible application). The wizard will suggest a correct setting automatically. If you are using the Windows Vista or MCE mediacenter, the MegaEPG will produce a native MCE programs guide. Otherwise, it will produce a guide in the standard XMLTV format.
Choose how many days you want the guide to be downloaded for. It is not recommended to specify too big (more than 7 days) or too small (less than 2 days) number.
If the application type is XMLTV, the wizard will ask you for output XMLTV file path (this page will not appear for the Windows Vista/MCE users). The path depends on the PVR/mediacenter application you are using. Please consult with your PVR application documentation.
Choose your TV provider (e.g. Yes or Hot, the providers' list depends on the country). Knowing the TV provider is important for correct TV box channel numbering and control.

Choose the script(s) that will download the TV guide for you. A script actually defines the Internet source to read the guide from.

You can choose more than one script: the MegaEPG will invoke a next script in the list if the previous one has failed to download a day guide for a TV channel. The script order is important and the 1st script in the list has the highest importance since it'll obtain the most of your programs guide.
Currently, the MegaEPG distribution comes with the following scripts:

Script name	Country	Description
tv2day_he	Israel	Hebrew EPG from the YES provider
tv2dayQ_he	Israel	Another Hebrew EPG from the YES provider. Works much faster than tv2day_he but does not obtain the original name and production year
hot_he	Israel	Hebrew EPG from the HOT provider
tv2day_ru	Israel	Russian EPG from the YES provider
vista_he	Israel	Hebrew EPG from the Microsoft Vista EPG service. Read more about the script before use!
ynet_he	Israel	Hebrew EPG from the Ynet site
walla_he	Israel	Hebrew EPG from the Walla site. (Not recommended, their schedule is often incorrect/incomplete)
dstv_za	South Africa	South African DSTV satellite provider

Select channels you want to download the guide for.

Add the channels from the All channels window to the Selected channels window. Multiple channels selection is available. The order in the Selected channels can be arranged then by clicking on the Up and Down buttons. The channel order defines the EPG displaying order in many PVR applications.
After selecting required channels, you can select the show (program) descriptions download method for every channel. Stay on a channel in the Selected channels and choose either Full Description, Smart Description (the default) or No Description radio button. Downloading the show(program) descriptions is a very time and resource-consuming process, therefore it is very important and highly recommended to understand and choose the optimal settings for you. Show descriptions download types are:
- Full Description - Show (program) descriptions will be unconditionally downloaded for the channel.
- Smart Description - Show descriptions will be downloaded for the channel only at the morning/evening prime-time, weekends and holidays (prime-time and holidays are defined on the next wizard page). This type is one of the most powerful features of the MegaEPG. It lets you enjoy complete show descriptions at the time you need them, still dramatically saving the downloading time and resources.
- No Description - Show descriptions will be not downloaded for the channel.
Please note that the Smart Description is the default description downloading method! You will need to define weekends and morning/evening primetime (see below). If you are not satisfied with the Smart Description method (e.g. you don't have fixed weekends and miss show description too often), then change all the selected channels to Full Description.
Note, that if you are using a complete provider script (one that downloads a complete available EPG at once, e.g vista_he), you may select no channels at the channels page and it will mean "Download all available channels the script is able to provide". For a regular script at least one channel must be selected.
Choose whether you want to use the Cache system.

It is recommended to enable the cache because nothing else reduces the guide downloading and processing time like the cache does. There are three cache modes: Always, Always except today and Never. The fist mode instructs MegaEPG to use cached guide information whenever possible. The 2nd mode uses the cached guide for tomorrow and subsequent days but today's guide will be downloaded from the Internet unconditionally. This mode is the most effective both for the EPG accuracy and time/resources consuming.
The Never mode disables the caching system. Disable the cache only if there are permanent problems with incorrect information about TV programs and frequent schedule changes.
If the selected script(s) requires Smart Description downloading, the "Weekends and prime time" page will be displayed. Choose weekends and prime times for the smart full descriptions download.

Channel(s) with Smart Description download type will get a complete description for the program shows on the defined weekends, holidays (defined in the holidays.dat file, or on regular days within the time range defined by the morning and evening primetime. For example, if you work or study and usually don't watch TV till 20:00, you can save the resources required to download complete show descriptions for shows you don't get to watch. Note, that if the script does not require additional downloading/resources to obtain complete descriptions, this page will not be displayed.
Enable the IMDB filter if you want to use the IMDB rating for selected movies.

The filter will try to obtain the IMDB rating for the guide movies if enough information exists for the movie. Minimum votes number defines the minimal amount of IMDB voters for the movie in order to assign it a stars rating. The default value is 1300 which means that a movie will be assigned a stars rating only if more than 1300 people voted on IMDB for the movie regardless of the rating itself. Min. rating to assign stars numbers define the minimal IMDB ratings for 4,3,2 and 1 stars respectively. They should be float numbers (numbers with a decimal point). The deafault values for the stars rating are: four stars will be assigned for a movie with IMDB rating 7.5 and up, three stars - for a movie with IMDB rating 6.9, two stars for a movie with the rating 6.5 and one star for a movie with the rating 6.0. Movies with the IMDB rating less than 6.0 will not get a star rating. You can change the numbers according your own preferences.
Choose if you want to run the MegaEPG upon the setup is complete (yes by default).

It's highly recommended to run the MegaEPG right after the configuration step and have the guide ready before setting up the PVR/media application.
If you want the advanced setup (only for advanced users), select the Yes radio button. The advanced setup lets you choose the log level settings and setup the logging emailing system. To be completed...

The MegaEPG should be executed at least once a day in order to obtain the TV guide for the upcoming days. The MegaEPG does not have an internal execution scheduler mechanism. It's the user responsibility to schedule the MegaEPG execution daily, either by the PVR application (if it has this feature) or by the system's "Scheduled Tasks" (cron in Linux). Do the following steps to schedule the MegaEPG in Windows XP / MCE operating system:

Open the Scheduled Tasks by clicking on Start, Settings, Control Panel, Scheduled Tasks.
Choose Add Scheduled Task. The Scheduled task wizard will appear.
Click on Next button, then Browse and choose the MegaEPG program, e.g.: c:\Program Files\MegaEPG\megaepg.exe (Note: In Windows MCE the path will be c:\Windows\ehome\MegaEPG\megaepg.exe).
Choose the Daily radio button. Click on Next.
Select time to start and ensure that the "Every Day" radio button is selected.

The time depends on your PVR application. Please refer to your PVR documentation and ensure that the MegaEPG is executed before the PVR application tries to accept the new TV guide.
Enter your user name and password. Use the same username and password you use to log on into your Windows system.
Check Open advanced properties checkbox before clicking on Finish button!
In the advanced properties choose the Settings tab and check the Wake the computer to run this task checkbox.
Click on Ok. The scheduling is now complete.

After installing, configuring and scheduling the MegaEPG, you should configure the Windows MCE to start getting native guide from the MegaEPG. Do the following:

Run the MegaEPG to obtain your first MegaEPG guide. Ensure that it has finished without errors. Don't forget this step!
Go to MCE Settings, TV, Guide, Set Up Guide Listings.
Enter your ZIP code or four zeroes as the postal code: 0000 (or 000000, depends on the country's postal code length)

Click on Next.
You should get a successful message that the provider information has been downloaded:

Click on Next.
Select MegaEPG provider and click on Next.
Windows MCE will now download and import the guide data.

After installing, configuring and scheduling the MegaEPG, you should configure the Windows MCE to start getting native guide from the MegaEPG. Do the following:

Run the MegaEPG to obtain your first MegaEPG guide. Ensure that it has finished without errors. Don't forget this step!
Go to Vista Media Center's Settings, TV, Guide, Set Up Guide Listings:
If the region is not Israel choose No, I want to select a different region, then select Israel:

Click on Next
Choose No when asked to download new options

Click on Next
Answer Yes to Do you want to use the Guide?

Click on Next
Enter your ZIP code. The MegaEPG accepts any ZIP code: enter your real ZIP code or four zeroes: 0000

Click on Next
Choose MegaEPG provider and click on Next:

Click on Next
The new guide will be downloaded. When completed, you will see the "Done" screen:

Click on Finish

This section describes how to configure the MegaEPG manually. You may also refer to the megaepg.ini.sample file for some settings explanations. A basic configuring of the MegaEPG is not too hard. Below are a few working megaepg.ini file examples.

This is the minimal configuration file for a HOT user with MCE/Vista that wants a 2-day EPG for three selected channels from the HOT site. Descriptions will be fully downloaded for every show/program.

You may also use your set-top-box channel numbers instead of symbolic names:

Now lets define the Smart Description downloading. Suppose that we want the descriptions be downloaded only for the shows/programs from 6:00 till 8:00 at mornings and from 18:00 till 23:00 at evenings during workdays and on weekends (Friday and Saturday) be downloaded for all programs. We'll add the new settings to the end and will change grab_full with grab_smart.Our new configuration file will look as follows:

The MegaEPG contains the TV channels definition file in Data\channels.xml. The author tries to keep the channels definition as updated as possible, however providers may add and remove channels frequently. If you discovered a new/deleted/changed channel, please write a note to the author to make the required changes in the "official" channels definition file. You may also edit your copy of the channels.xml file and make the change without waiting for an updated official one. To add a channel write a new channel tag within the channels hierarchy. For example:

The id must be a unique number, the name will be used to refer to the channel in the MegaEPG configuration grab_... settings. The list of available categories is specified at the beginning of the channels definition file. displayname defines the name of the channel as it will be displayed in the EPG grid. You can define multiple displaynames for different languages. tuning defines the set-top-box channel number. If the channel applies only to the satellite or cable provider, you should specify only sat and cab providers respectively. If the provider replaced a channel with another one, do NOT add the new channel! Just change the name and displayname to match the new channel, but keep the same id. In any case make a backup of the channels.xml file before any change!

The following table defines the (almost) complete list of the megaepg.ini file settings. Please use the keywords only if you fully understand what you are doing.

Keyword	Description
loglevel	Logging level: critical (less output), error, warning, info, debug (lots of output, beware!)
logpath	Logging path. Specify the directory name where log files will be stored. The default is MegaEPG\logs.
days	Specifies the number of days the guide should be downloaded for. The default is 1
script	Specifies the script name (or a comma separated list of scripts) that will obtain the guide. The script(s) must exist as a Python (.py) or compliled Python (.pyc,.pyo) files in the Scripts directory
language	System language that applies to the channel names and categories. Possible values depend on the country, e.g. he_IL, ru, en
tvprovider	Defines the television provider for correct channel mapping. Available values depend on the country and will be a value from the providercodes, e.g. sat for the satellite TV provider, cab for the cable provider
grabchannels	Comma separated channel names to grab the EPG for. Channel names are defined by the "name" attribute (see the data/channels.xml file). Example: grabchannels=isr2,yes1
grab_full	Comma separated list of channel names to unconditionally download show descriptions for. The channel names must be a subset of grabchannels list. You can use *all* keyword instead of naming all the channels.
grab_smart	Comma separated list of channel names to use Smart download of show descriptions for. The channel names must be a subset of grabchannels list. You can use *all* keyword instead of naming all the channels. If you use smart show description downloading (much recommended), don't forget to set weekends, morning and evening primetime.
grab_nodesc	Comma separated list of channel names to take no show descriptions for. Only basic information will be downloaded for these channels. The channel names must be a subset of grabchannels list. You can use *all* keyword instead of naming all the channels.
epgformat	Output file format. Use xmltv for the standard XMLTV output file. Use mce for the Windows MCE/Vista native guide format.
outputencoding	Output XMLTV file encoding. Default is utf-8 which is highly recommended. Change the encoding only if your PVR application limits the XMLTV file encoding for some reason.
xmlpath	Output path for the XMLTV file. Default is the MegaEPG/xml directory. Change it only if your PVR application requires a specific path it expects the XMLTV file at. Don't use the keyword in Windows MCE.
xmlfile	Output filename for the XMLTV file. Default is epgdata.xml. Change it only if your PVR application requires a specific filename. Has no effect in Windows MCE.
delay	Adds a delay between channel downloads. Specify the number of seconds to wait before starting downloading the next channel. Using of the delaying is highly appreciated, it reduces the load on the provider's servers. Example: delay=1
usecache	Specify 1 to use the cache system, 2 to use it always but today or 0 to disable it. The cache system allows reusing of the previously downloaded information without the need to download it again. The default is "disabled"
morning	Defines morning prime-time range. "Smart" show description download will get show (program) description for programs that start within the time range. Format: h:mm-h:mm. Example: morning=7:00-9:00.
evening	Defines evening prime-time range. "Smart" show description download will get show description for programs that start within the time range. Format: h:mm-h:mm. Midnight is defined as 24:00. You can use 25:00 for 1am, 26:00 for 2am and so on. Example: evening=18:00-26:00
weekends	Comma separated days of week (1-Sunday) that will be treated as weekends (smart show description download will get show descriptions for the whole weekend). Example: weekends=6,7 defines Friday and Saturday as weekends.
holidays	Holidays filename. The default is holidays.dat
tzoffset	Time zone offset. Obsolete. The parameter will be used only if the MegaEPG can not get the system timezone. Format: +0200 for UTC+2 or -0100 for UTC-1, etc. Example: tzoffset=+0300 for Israel with daylight time. tzoffset=+0200 for Israel Standard time
channelicons	Specify 1 to enable channel icons, 0 - to disable it (the default). Note, channel icons are applicable in certain PVR applications. Do not enable them if your application does not support channel icons in the EPG. If you enabled the channel icons, specify also "iconpath" parameter.
iconpath	Specifies the channel icons directory if "channelicons" is enabled. Icon for the channel must be a JPEG image file with the channel attribute name and .jpg extension, for example: isr2.jpg, yes1.jpg. Important! The icons directory must be specified in a URL form. Example: file://d:/channels/
originalname	Defines where to put original show names. Use subtitle to make the subtitle XMLTV field keep the original name, use desc to add the original name to the show detailed description. Setting of the parameter should depend on the PVR application you are using and the way it displays EPG on the screen. "desc" looks nicer in Windows MCE while "subtitle" looks nicer in SageTV (at least in version 4).
episodename	Defines where to put the episode numbers. Use subtitle to add the episode number to the subtitle. Use episode to specify it in the standard episode XMLTV field. If your PVR application does not parse the episode number field, use "subtitle"
chanid	Changes generation of channels id in the XMLTV file. name generates channel short names as id (the default); name+display generates channel short names as id and also generates the 2nd <display-name> with the channel STB number to let some PVR application map channel numbers to the channel name automatically; stbid generates channel STB numbers as id.
http_port	Specifies the TCP/IP port for the MegaEPG Vista service. Ignored in other operating systems. Default is 19679. Change only if the port is busy by another network service/application.
filter	Specifies the filter script name (or a comma separated list of scripts) that will process every downloaded programs in the guide in order to get additional information about the program. Currently the only available filter script is imdb. The script(s) must exist as a Python (.py) or compliled Python (.pyc,.pyo) files in the Scripts directory
country	Specifies the ISO country code (e.g. IL for Israel, ZA for South Africa). The parameter is normally set during the MegaEPG installation. Do not change the parameter
channels	Specifies the name of the channels XML file. The parameter is normally set during the MegaEPG installation. Do not change the parameter
languages	Specifies the available languages codes (e.g. en, he_IL, ru). The parameter is normally set during the MegaEPG installation. Do not change the parameter
providers	Specifies the list of available TV providers' names available in the country. The parameter is normally set during the MegaEPG installation. Do not change the parameter
providercodes	Specifies the list of TV providers' codes (e.g. sat, cab). The parameter is normally set during the MegaEPG installation. Do not change the parameter
sockerr_retries	In case of a socket error caused by the network not ready after the system wakes up from sleep/standby to execute the scheduled megaepg, this parameter specifies how many connection retries will be done together with the sleep (see sockerr_delay) in order to ensure that the network is operational. Default value: 1. "Network not ready" can happen in a system that has a wireless network adapter or a slow DHCP server/lengthy IP address assignment.
sockerr_delay	Specifies timeout in seconds to wait before retrying the connection in case of a "network not ready" socket error, see the above parameter. The default value is 30.

The following keywords are yet to be described: agent, cleanold, mail_enabled, mail_user_to, mail_user_from, mail_smtphost, mail_auth_required, mail_auth_user, mail_auth_pass computername, keephtml, keepmce, nolangattr, simplestxml.

The EPG script from the Microsoft EPG service provides full EPG data for Israel YES provider and partial EPG data for HOT as a part of the Vista Media Center support. Please read and agree to the following before using the script:

The script is capable to obtain a complete EPG for all the available channels. Below is an example of the minimal megaepg.ini file for use with the script (do not use the epgconfig Wizard since it does not currently support "all channels"):

Choose a correct tvprovider (should be cab for HOT). The tzoffset parameter is set to GMT+3 due to early DST introduction in Israel. Later on the parameter should be removed (or set to +0200) to keep the standard GMT+2 timezone.

IMDB is the only filter script that is currently distributed with the MegaEPG. The script obtains IMDB ratings for the guide movies. Please use the epgconfig wizard for the script enabling/configuration. Advanced users may use the manual configuration: edit the megaepg.ini configuration file with your favorite editor and add the following:

Add the filter=imdb line to the global section.
Add new section called [imdb] to the end of the file (if the section still does not exist). Add minvotes and stars parameters to the section, so it'll look similar to:
[imdb] minvotes=1300 stars=7.5,6.9,6.5,6.0
Adjust the parameters' numbers to your personal preference. stars must contain four float number separated by a comma. If you do not want the IMDB rating to be added to show descriptions, use addtodesc=0 parameter in the [imdb] section. Us addtodesc=2 to add the rating in front of the show description, otherwise it is added at the end of the description.

Install the wine package if it's not installed. MegaEPG was tested with wine version 0.9.36, there is no warranty for other versions.
Download the megaepg_setup.exe to your home directory and run the megaepg setup by executing: wine megaepg_setup.exe
Go thru the Setup wizard, don't change the installation path. Continue with the Configuration wizard if you want (although you can create the configuration file megaepg.ini manually in the directory ~/.wine/drive_c/Program Files/MegaEPG
Now you can run or cron it as usual. The command for execution: wine "c:\\Program Files\\MegaEPG\\megaepg"
The result EPG file is in ~/.wine/drive_c/Program Files/MegaEPG/xml/epgdata.xml. Copy it to appropriate location or provide the path to your TV application.

The method was tested on Fedora Core 6 distribution. For FC6 the wine package can be easily installed with the Package Manager (Add/Remove Software from the Gnome menu). Choose the Search tab, type "wine", check the wine-0.9.36 and install it. The package manager will automatically install the dependencies.

The software is provided "as is" without warranties of any kind, either expressed or implied. The author shall not be liable for any direct, indirect, consequential or incidental damage arising out of use or inability to use the software.
Redistribution is allowed for the unchanged distribution package only. Repackaging of the software in any form or making the package a part of any other software (bundling) is strictly prohibited. Reverse engineering or modifying parts of the software or its documentation is strictly prohibited. The digital signature of the package must remain valid.
The software is licensed for personal use in a home/private environment. Commercial use of the software or distribution of the data produced by the software as a service (either commercial or free) is strictly prohibited.
EPG is a copyrighted data in most cases. You agree to comply with the copyright law and the EPG rights holder regulations. Microsoft EPG service in Israel is provided for Windows Vista customers only. In order to use the Microsoft EPG service scripts you must have a valid Vista Home Premium or Vista Ultimate license. The author shall not be liable for a misuse or illegal use of copyrighted data.

For updates, bug reports, questions and discussion please visit http://pvr.co.il. Send your ideas and suggestions to pvr@pvr.co.il.

MegaEPG

About the MegaEPG

Installation

Configuration

MegaEPG execution scheduling

MegaEPG Windows MCE setup

MegaEPG Windows Vista setup

Manual configuration

Adding or changing channels

All Ini file settings (for advanced users)

Microsoft EPG script setup (for Israel only)

Filter scripts

MegaEPG under Linux

Licensing

And finally...