How to make "Get Missing Components" actually work

First off, I have posted most of this info here there and everywhere on this forum but it was always a threads I hijacked. Maybe it will get more visibility in a dedicated thread.

Some of you may know that the "Get Missing Components" button relies on the rfcmp having a proper download URL embedded in it. You can see this URL in the ModMgr. Most components are either missing this link completely or it points to the modders home page or something like that. It seems no modder understands what this URL was really designed to do. You will notice all of ISI's content have this URL:
http://rfactor.net/downloads/getmod.php

The way GetMod works is rF2 appends ?sig=<sig> to the end of the URL. <sig> being the sig of the rfcmp that rF2 wants to download. This sig is the sig that you see in the ModMgr. ISI's GetMod.php reads in the sig parameter and then looks in a database for the actual download URL and then redirects rF2 to that URL. This is the ultimate way to handle it because all your rfcmp's can have the same URL and the actual download URL can be handled by your GetMod webservice. So if you want to move your rfcmp to another webhost you don't have to repackage your rfcmp to include a new download link. All you would actually have to do is update your database with the new URL.

The problem is ISI hasn't made this database open to the public for people to register download links for their own content. So I made a system of my own and made it public and free for anyone to use. You can register for an account on my service here and then you can register your components and download URLs:
http://getmod.srrs-racing.net/registration.php

This service isn't costing me any money and I have no plans to ever take it down or request money for using it.

I also released the source code for this web app if there are any admins out there that want to host it on their own server. I released it in this thread:
http://isiforums.net/f/showthread.p...onents-Feature?p=188568&viewfull=1#post188568

Some things to keep in mind is that rF2 only works with DIRECT download links. It can't be a URL that brings you to a web page with advertisements, captas or anything like that. It has to be a raw link that when clicked in a browser starts a transfer without asking any questions. This rules out almost all free file sharing services unfortunately. The other limitation is rF2 doesn't support HTTPS. Some webhosts let you use HTTP but then it just redirects to HTTPS. These host also will not work.

I will edit this post with info on any free filesharing hosts that I discover that will work with rF2's GetMod system.

Supported File Sharing hosts:
Dropbox: 20GB per day for free accounts and 200GB per day for pay accounts. If you go over the limit they will block all your public links for a few days.

If anyone has a favorite filesharing service let me know and I will research if it will work with GetMod and I will edit this post with details. Another thing I may add to my GetMod.php service is some sort load balancing so you can host your rfcmp on multiple sites and when rF2 hits my service it will rotate which URL it responds with. Dropbox for example only allows 20GB of downloads per day on a free account.

WARNING: MultiCmps are not supported by GetMod. MultiCmps are simply for distributing a bunch of rfcmps in one big file. To use a MultiCmp you will have to extract the individual rfcmp files from the MultiCmp and then host and register them individually. So don't come to my service and register one MultiCmp and not be able to make it work and then give up altogether. I see a ton of accounts on my service with one Multi registered and that is it. I suspect these people start right off doing it wrong and then give up completely.


I released a new tool the interfaces with my GetMod webservice and makes registering components very simple. No more repackaging components and all that stuff. You can get it here:
http://isiforums.net/f/showthread.php/19702-GetMod-Manager?p=271771&viewfull=1#post271771

 

Before anyone asks. Mediafire will not work. They require cookies to be used when downloading. If you get a direct link and then pass it to another client, that direct link will not work because they don't have the proper cookies to authorize the download. For Mediafire to work, more logic would have to be added to rF2's HTTP client.

 

Well done Noel, I hope modders use it.

 

The "Get Missing Components" feature was made public in build 218 and I did lots of testing to figure out exactly how it worked the day it was released. My discoveries have been out here all this time and so far only one modder has taken advantage of it and that is PetraGTC. That is mostly because I contacted her directly with instructions on how to do it. Actually all she is doing is including the URL to my service in her rfcmp's and I take care of adding the Sigs and URLs to the database. But as I said in my post, I posted my finding in several threads all over the place. Maybe posting it here in a dedicated thread will do the trick.

 

This question is for ISI directly: Would it be possible for ISI to offer hosting for approved mods so that GetMod would refer to the official ISI server hosting the file? This would avoid the problems with dropbox getting overloaded (it is very limited), web hosts changing domains or closing up shop. It'd be great if a GetMod URL that is made today still worked in 5 years because ISI is still maintaining the file host.

 

Guy, that is the beauty of how ISI dreamed this system up. The URL in the rfcmp points to a database that simply redirects to the actually URL where the rfcmp is hosted. If that host goes down you just go back to your database and update it with a new link. I do however wish ISI would make their inhouse GetMod.php service open to the public.

As I said above I will be adding a load balancing feature to my GetMod service so you could host your rfcmp on say 5 different Dropbox accounts and then register all 5 of those links in the database and it will rotate through the links one at a time effectively giving you 100GB of bandwidth per day. The problem is DB doesn't give you 20GB per day per file, it is 20GB total per day. So if you have an 800MB mod busted up over 14 rfcmps it will still only give you 25 downloads per day. But in the case of Petra's mod, she only updates one car here and one car there so we would only be looking at a 60MB (at the most) per download. So on a day that only one car is updates that would allow 333 people download the update. Another thing my system could do is load balance for the links a modder may post on a forum. They could host there rfcmp on 5 different DB accounts and then post just one link on the form and you don't have to worry about people always clicking on the first link in the post.

 

Kind of funny, I just looked at maybe using bitly.com as a GetMod database. If you create an account at Bitly (free) you can create short URLs which you can later go back and change the target URL on. So for an rfcmp you would go to Bitly and put in some bogus URL just to get a new short URL. You then take that short URL and past it into the URL field when packaging your new rfcmp. Then you host your rfcmp somewhere and once you know what the direct download URL is, you go back to your Bitly account and change the target URL to the actual download URL. One of the cool things about Bitly is they already have a nice GUI for managing your links (unlike mine) and they also keep click statistics. How cool would it be to know exactly how many times your mod has been downloaded?

I am going to repackage a track and test this concept out and see if it works with rF2. I already did a quick test where I took a bitly link and added ?sig=blablabla to the end of the URL and it just ignores the extra stuff. I tested this with TinyURL and it didn't like the extra stuff. This may work. I will report back.

 

I thought this was the point of the $84 dollars for lifetime subscription or $12 dollars a year for multiplayer.

 

Nope... Bitly is a no go. I haven't watched the raw traffic with Fiddler yet but my guess is Bitly does a bunch of junk before handing off the final URL which breaks rF2. Darn, that would have been a handy fix.

 

They never said they were going to host mods. They said they would release free content and host the matchmaker.

And please lets not turn this into a thread about how $12 a year is too much to ask for content updates. That crap can be left on VirtualR. Hahaha.

 

I didn't mean that as flamebait- I meant it as in, would ISI consider such a thing- which is why I directed the question directly to them. This has nothing to do with the money I spent for the game or anything, and it would only be for mods ISI approves (the ones that pass the Hammer Test). It'd enhance their product and make pubby racing much better- right now it is difficult to join a pubby race.

 

Yes that is a valid question and my response wasn't directed at you. Hosting isn't a cheap thing but they could at least make the GetMod database they use for their own content available to anyone. The traffic it generates should be minimal.

 

Okay.... Found a way for anyone to host with GetMod support without haven't their own webhost or using my service.

I found a URL shortener (Snipurl.com) that lets you sign up for an account so that all your short URLs can be edited, tracked and deleted. The idea is that you create a dummy short URL as a place holder. You then paste that URL into your rfcmp. Then you upload the rfcmp to Dropbox, you then get a dirrect link using the method I outlined in the first post, then you go back to Snipurl.com and edit your placeholder URL and paste your Dropbox URL there. After that, anyone who hosts with your rfcmp will have full GetMod support. If for what ever reason you move the rfcmp to another host, you just go back to Snipurl.com and update your short URL with the new host.

Maybe I will make a video tutorial of this method.

 

If anyone wants to see GetMod in action. I setup a test server running the Camaro at Mid-Ohio. The Camaro already supports GetMod but I repackaged Mid-Ohio so that it would have a proper GetMod URL. The submit ID hasn't changed so if you already have it installed this test isn't going to prove much. To truly test it, uninstall Mid-Ohio and remove it from your packages folder.

This test is using Snipurl.com and hosted on Dropbox.

My server is named "Noel's Test".

 

<Deleted>

Refer to this post:
http://isiforums.net/f/showthread.p...-actually-work?p=206990&viewfull=1#post206990

 

Thanks!!

I just tried it and it works great!

 

Great Noel.

 

I found an even better URL Shortener for our purposes. MultiURL.com. The advantage of MultiURL.com is it lets you set more than one site as a target and then when you go to the short URL it will randomly deside which target to go to. So if you have your rfcmp stored on multiple Dropbox accounts you can assign all of those links to your short url and then when rF2 does a GetMod it will randomly go to different URLs which will effectively load balance across each dropbox account.

MultiURL also keeps click statistics for each link and total clicks.

So it looks like a combination of Dropbox and MultiURL.com is the best solution if you dont want to bother with my service or host your own getmod using my PHP source code.

 

ISI can't host unlicenced content.... the legality would prevent it! Mods on established series are already in a grey area as it is.

 

Good point. They may not even want to allow use of their GetMod database even though they wouldn't actually be hosting. Very similar to Torrent search engines. They don't really host the movies/software but they help connect you to them.