Multiple concurrent connections (2+) fails

Hi Emanem,

I've never experienced a problem like yours and I'm not sure whos fault it is. An ISP that tampers with the transfered data seems rather unlikely to me.

Does the message log in the Help menu say something about the problem? LottaNZB isn't responsible for the actual downloads, but uses HellaNZB for this. HellaNZB is, being unmaintained for over two years, known to have some bugs. This is why I suggest you to give another NZB application like SABnzbd a try and check whether the problem happens there too. If you use Ubuntu, you can simply install the "sabnzbdplus" package and launch it by running "sabnzbdplus" (Alt-F2 or in the terminal) and control the program from within your webbrowser.

Hope this brings us closer to an answer.

Hi Severin,

I will do that later on, but I am so tempted to write a small C++ NZB daemon (given the simplicity of NNTP protocol) and test it to understand what could happen...
Really I think it's very strange as well, but when I set more than 2 connections I get corrupted files without errors on the log.
Btw I'm using the biggest ISP in UK ... and my connection works ok I'd say.

Cheers,

Implementing the NNTP protocol on your own would certainly be an interesting thing. Note that there is already a C++-based NZB grabber called nzbget: http://sourceforge.net/projects/nzbget/

Thanks for the suggestion, I'll try with NZBget and maybe write some C++ code... I'll let you know what I find.
Cheers,

Hi Severin,

I've developed a small multithread/multiconnection NZB downloader to properly test/debug my current issues.
What happens is the following:

1) my current ISP allows max 2 connections to its NNTP server (authentication is not needed)
2) on a third (or more) connection it will require authentication; not having a valid username/password it will drop the connection
3) the NNTP server my ISP is using is badly implemented (I'd say a bit cr*ppy). When you close TCP/IP connections on a client side (using the QUIT command as well...) it will take 0.5~1 seconds on its (server) side to realize the connections are closed (I'm using QUIT command, then 'shutdown' and 'close' to properly close the TCP/IP socket).

As seen my ISP (biggest UK...) is relying on news provider X (I can read the proper name in the welcome message), I fear that they have a limit of max 2 TCP/IP connections coming from same IP on my ISP's network.
Anyway I'll ask tech support if username/password are required, but, as seen as with 2 connection everything works (not perfectly - see point 3) I guess authentication is not needed and this is just a limit; I think news provider X is forcing authentication just to drop 3+ connections coming from the same IP.

Regarding point (3), I discovered that with my C++ multithreaded NZB client: even if I gracefully close connections, unless I put a nanosleep of 500+ ms the server will think I have connections alive thus asking for 'fake' authentication once I ask for new data. This is sooo sad...

But the most sad things of all is that even if I use 2 connections only for the whole download of segments (so I do not create-use-close a connection for each segment) after I've used a connection to ask for a NNTP ARTICLE/BODY, the connection will be kept alive by the server, but the server itself won't respond to future requests.
At the end of the day I'm forced to use the create-use-close connection for each segment, every time opening and closing TCP/IP connections (why do we have to waste bandwidth and resources?!?!).

At the end I've experienced that news provider X:
A) they use 'fake' authentication to block more than 2 connections coming from the same IP
B) their server is cr*ppy: it doesn't manage neither the NNTP protocol, neither the TCP/IP sockets in a proper way

Using nanosleep to fix even the fact their server can detect a closed connection when done gracefully is a total shame...I fear their server is an old monothread 'select' like server... what a shame...
I'll let you know about what my ISP say regarding possible username/password but I don't think they'll say anything.

Instead I'm really disappointed towards HellaNZB. That piece of software doesn't report any error.
While LottaNZB is ok, why don't you rely on a properly written C/C++ core to perform the download?

I hope all this information is/will be useful!
Once a polish a bit my code maybe I'll publish it under the GPLv3.
Cheers,

Please scrap this:

But the most sad things of all is that even if I use 2 connections only for the whole download of segments (so I do not create-use-close a connection for each segment) after I've used a connection to ask for a NNTP ARTICLE/BODY, the connection will be kept alive by the server, but the server itself won't respond to future requests.
At the end of the day I'm forced to use the create-use-close connection for each segment, every time opening and closing TCP/IP connections (why do we have to waste bandwidth and resources?!?!).

There was a glitch in my code, now I've fixed and I can use 2 shared connections.
But all the rest of my answer holds true.
Cheers,

As promised here:
http://mynzb.youlink.org
Is my GPL v3 NZB client.
Let me know what you think,

Cheers,

Hi Emanuele,

sorry for not getting back to you earlier. Unfortunately, I was in bed with fever during the past week.

I must say it's interesting to see what you've come up with to work around the poor NNTP implementation of your Usenet provider. I haven't tried mynzb yet but I'm sure it does its job well, efficiently and just in the spirit of Unix programs.

I agree that a good NZB grabber should at least report a useful error if the Usenet server it's connected to misbehaves. As I already said, HellaNZB has lots of such flaws (although it works fine for the majority of users).

I already wrote that we're migrating to sabnzbdplus. While it's still written in Python and thus potentially memory-intensive and slow, it's at least being developed actively, has an extensive API and lots of features. Of course, LottaNZB could also have been implemented in a language like C, C++ or Vala.

When I started working on LottaNZB, I didn't have any real experience in C or C++ programming (now I do, thanks to my studies in Computer Science) and I'd love to work on a project in these languages, but I must still say that Python is a very expressive and flexible language that comes in handy when used with care.

Congratulations again on your work!

Hi Severin,

I'm glad to hear you are starting developing in C/C++.
Personally I think that's the best language as now, because (C++) gives you flexibility and power other languages don't give you (eg. you can both abstract and go very near to the CPU at the same time).
Anyway, after more than 10 years of experience maybe my opinion is slightly biased ( :-P ) but if you want you can take a look at the sources of myNZB ( http://mynzb.youlink.org ) and again, you feel free to ask me questions about it.

When I have to write prototype software I usually use Perl or straight C++.
Probably the major issue with C/C++ is that you don't have a pre-defined library for everything... so at first hand is hard to craft some prototype code, but this is the best thing, because it allows you to do different things with potentially different libraries.
Just think about Java/C#... if for example the only (provided) JPEG library would have a particular bug, you would find yourself in the same situation as C++ (you have to write your own), but with C++, as seen as there's not a pre-defined library, you can choose between 3 or more libraries to perform the same operation (Intel, IJG, ...).
Plus, not reasoning by forced path, will allow your mind to be more flexible.

Anyway, the good thing of being open source (GPL v3) is that you can use/amend/modify it without worrying about.
Really, if you want, have a look at the sources: those aren't very complex! :-)
My email is at the bottom of the webpage.

Cheers,

Ps. I'm not against GUI, indeed for another project I'm using gtkmm (C++ version of GTK+), but for this kind of NNTP client all I want is the job done.