Archive for the 'Documentation' Category

Number your Result

Tuesday, July 11th, 2006

I assume you are by now a NMDC-search-hating person. All heil ADC’s search!

Yeah, I bit strong I know… But still. The NMDC-suckiness continues with the number of search results you recieve.

In NMDC, when sending a search, you will either get (from each user) 5 results if you are in Passive mode or 10 if you are in active mode. ADC have this restriction also. “Okay… Then what’s sucky if they’re the same?” Well, it’s a matter of which results you recieve.

In NMDC, there are no way to say “I want this, but not that”. In ADC on the other hand, there is; “Each filename (including the path to it) should be matched using case insensitive substring search as follows: match all AN, remove those that match any NO […] “. In laymans terms, “if I specify in the search that I want only file names (and paths) that have ’linux’ and doesn’t have ‘ubuntu’ in it (file name/path), I will only recieve results that have ‘linux’ in the file name/path and the sender of the results will have discarded all file names and paths matching ‘ubuntu’.”

So you dig through the archives, and find Filtering…, and think “hey! It says there I can filter out search results with -foo!”… Well, you’d be filtering the recieved results in NMDC. (Ehm, said ADC functionality, in DC++, doesn’t currently exist, mind you…)

If you don’t understand the problem; When you are sending a search in NMDC, the sender of the results does not do ANY filtering. He’ll send you the first 5 or 10 results said client can find. The client couldn’t care less if you don’t want ‘ubuntu’. But, in ADC, the sender of the results will know “hm, I can’t just blindly send results, this person has specifically said ‘ubuntu’ is not desired.”

Answer to Pop quiz: Downloading in ADC

Sunday, July 2nd, 2006

I think this quiz was a little harder than the previous, but nontheless we have a winner, and it is (again) poy. Congrats. You win a free trip to the public DC dev hub. (Blatant plug :D )

One of the reason I believe this quiz was more difficult was because I didn’t include the actual answer in the options. (Well, d)…) Another reason is because it (sort of) require that you have been paying attention.

The question was: Why can’t Jake download the file list or any other file from “john111/2″? Because Jake already have a transfer going from “john111″. You see, “foobar” and “john111″ is the exact same person, using the same client, but with different user names. I’ll soon get to why they are the same user.

The reason I chose this scenario with university hubs, is because users tied to a university often share IPs. This means that one can’t look at the IP for comparison if two users are the same. (If people wouldn’t have shared IP, this would be a nice clue as to why Jake couldn’t download.)

So, why are “foobar” and “john111″ the same user? They have the exact same CID. I wrote in the previous post that DC++ look at the CID to check if two users are the same. ADC natively require CIDs, meaning that DC++ doesn’t “artificially” create the CID of other users. In terms, CIDs are global, (or atleast are supposed to be) meaning that your CID is the same in one, two or 40 ADC hubs.

Since DC++ see that they have the same CID, it thinks “hey, same user, don’t start another download because I’m already downloading from that user”.

 

There is an issue with the scheme DC++ currently use - Most users won’t understand what a CID is (CIDs are infact displayed in the user list) or why DC++ won’t start another download (”what the… it’s not the same user! crappy program!”). There have been discussion on how one could notify the user, among them; treeview of each download (”click on a ‘+’ to expand the list of usernames”); displaying “username1,username2,username3″ (I think this would be the easiest - codewise); Have DC++ print “You are already downloading from this user. The userlist/file has been queued”; a download mini slot option (like the upload mini slot option - but applying only to certain files, like the file list). Can anyone think of more?

Answer to Pop quiz: Downloading in NMDC hubs

Friday, June 30th, 2006

Time for the answer to the Pop quiz: Downloading in NMDC hubs.

First off, I want to congratulate poy for giving us the correct answers;
Q: How do you get the three files simultaniously from that user? (With DC++ 0.674)
A: Alternative b); use a different user name in all hubs.

Q: How do you get the three files simultaniously from that user? (With DC++ 0.691)
A: Alternative a); use the same user name in all hubs and b) use a different user name in all hubs.
Why alternative b) with DC++ 0.674? Well, it all has to do with how DC++ treat users. Users are identified by nick name alone (in 0.674), meaning that DC++ think “hey, you can’t start another transfer just yet from ‘foobar’, you are already downloading from him!”/”hey, you can’t start another transfer yet to _yournick_, you are already uploading to him!”. The “problem” is that there is no notification (eg, a reprahasing of what I just wrote).

Why is this different in DC++ 0.691 then? Why is option a) also correct? Because DC++ has changed its identification scheme. Now, DC++ identifies users with nick name and the hub address. This makes downloading files simultaniously incredibly easier with DC++ 0.691. You see, when DC++ check if it can connect to users, it looks at the CID of the user (CID is this identification, it basically is “encode with the Base32 mechanism[hash with Tiger[nick name + hub address]]”), and yet the nick name for is the same in the various hubs, the hub addresses surely aren’t. This is terms mean that DC++ think “not same user, allow transfer”, and we have ourselves a simultanious downloads.
This new scheme meant (as you’ve noticed) that all users in the queue and the favorite users were gone. The reason is above. The users had no CID, meaning DC++ thought “no CID, I have no idea who these people are, better get rid of them”. Now, there could have beeen some form of “conversion” for the favorite users, considering DC++ does store the hub name for the users, but it does not (as far as I know, I haven’t checked) for the queue. (I am not going to do a patch or any tool. Forget about it.)
What do you think? Was the old scheme better or is the new one? (Ignoring the upgrading problems.)
I’m hesitant to answer that question myself.

Pop quiz: Downloading in ADC

Saturday, June 24th, 2006

The second pop quiz is as follows;
‘Jake’ using an unmodified version of DC++ 0.691. He’s connected to 2 (two) ADC hubs with the nicks “jake99″ and “mary77″. He’s looking for the file A. He find user “foobar” in one of the hubs who have the files he’s looking for. This user is also using an unmodified version of DC++ 0.691. He start to download the file, going in 20 KiB/s and everything’s fine. But, he start to think, “what will happen if the user goes offline? better I auto search for it to check if there’s more sources”. And he does, and find user “john111″. Jake adds the user to list of sources (through the right click command menu).

Interesting in things as Jake is, he want to see what other goodies “john111″ has in his share. Jake tries to get the file list. Nothing. Nothing happens. He checks the queue, DC++ has properly queued the file list, but isn’t connecting. Nothing appear in the transfer view. The transfer view columns are fine, Jake can see that he has two downloads going (one of them the file A) and three uploads.

 But there’s no error message in the queue or the transfer view window. He deletes the file list from the queue, and tries again to download it. Same thing. Now, why can’t Jake download the file list? “john111″ isn’t blocking uploads by the looks of it in the search window (Jake can see that “john111″ has 5 slots open out of 7). Jake and “john111″ is both connected with Active mode.

The two hubs are both resident on the same university campus, where a lot of people connect from. Meaning, a lot of people have the same external IP. (Jake find out that “john111″ has the same IP as “foobar” and one of Jake’s uploads, and he draws the conclusion that those three are all living on the same campus.)

Jake asks his friend in the other room to try to download “john111″’s file list (and other files), and it works fine. It also works for other users in the same hub. Jake asks “john111″ to change his nickname to “john112″. No change in connectivity. “john112″ tell Jake to try and search for file B and try to download it. Same thing happens with file B. File B is now queued, but there’s no connectivity and no error message.

So; why can’t Jake download the file list or any other file? (File A has been downloading the entire time, now at 34 %.)

Because
a) Jake is using different user names.
b) there’s a bug in Jake’s copy of DC++.
c) “john112″ and everyone else is lying, at Jake’s expense.
d) none of the above… (fill in what the reason is)

It would be neat if you have an explanation why you chose that particular option, but you don’t have to; I’ll explain later which option/answer is correct, and why.

Pop quiz: Downloading in NMDC hubs

Saturday, June 24th, 2006

To increase the interaction between the blog and our blog readers, I thought we could have a “pop quiz”. I’ll also be so kind to have multiple choices for you to pick from.

Today’s pop quiz;
You are using an unmodified version of DC++ 0.674. You are connected to 3 (three) NMDC hubs with no relation. (They aren’t connected through a “network” that is.) You are looking for the files A, B and C. You find user “foobar” in the three hubs (the same user accross hubs) who have all the files you’re looking for. This user is also using an unmodified version of DC++ 0.674. Now, the question is; How do you get the three files simultaniously from that user? Speed is not a matter, neither is active vs passive mode and there’s no hub restrictions or slot restrictions from “foobar”.
Do you
a) use the same user name in all hubs
b) use a different user name in all hubs
c) call two friends and asks them to download file B respectively file C while you download file A. You then walk over to their homes and burn the files on a CD.
d) completely ignores the question I asked and download the files in a row (that is, not simultaniously).
e) say I’m stupid because “foobar”’s copy of DC++ will use the IP to not allow you to download them simultaniously.
f) none of the above, you… (fill in what you do)

Now, the same question, but with the difference that you and “foobar” are both using DC++ 0.691.
It would be neat if you have an explanation why you chose that particular option, but you don’t have to; I’ll explain later which option/answer is correct, and why.

SCH proudly presents $Search

Monday, June 19th, 2006

Ah, yes, today is a “why the NMDC protocol suck and ADC is so great”-post. And today’s protocol bit is brought to you by $Search (paid and sponsored by SCH in ADC) and the file types used. (You should have a look at the Search types post I made a while ago.)

Now, on to the (NMDC) protocol bit. All of the search types are “hard coded” into the protocol, meaning you will need to upgrade other clients to respond to your file type searches. This is unfortunately not enough of the NMDC sillyness. When you pick a certain file type in DC++, DC++ will change the caption of your choice (”Audio” eg) and to a number. This number is predefined (as the wiki post says) and cannot be changed (because otherwise, you’d recieve wrong search results or not get any - depending on what you change the number to). This basically means that you cannot tell DC++ (code wise, considering there is no way to do it in the GUI) to “search for _filename_ but only include audio extension AND video extension”. This is because there is no way to combine the different numbers. Now you’re thinking “but can’t you add them and the send that number out?”, and sure, you can do that. But you won’t get the result your after. Eg, “Audio” is 2 and “Compressed files” is 3. Now, if you add these, they (of course) become 5. Now, the problem is that 5 is “Executables”, making all of the other clients think you’re after “executables” and not “audio” and “compressed files”.

“Sigh… Every time you talk about NMDC, I hear something bad… Fine. So, what’s so good with ADC then?”

Ah, my dearest Watson, it is very simple. In ADC’s SCH, one can specify a “EX” (”extension”) when searching, and the responding client have to match files with atleast one of those EXs. This means that we can specify completely what kind of file types we want - even if the other client hasn’t hard coded any file extensions in its client).

Me, myself and Unsigned

Monday, June 12th, 2006

One of the problems when creating software is to predict and know what kind of input people will give when they fiddle around in the settings of the program. DC++ no less also has this issue. Though, while it’s easy to picture (as a programmer) what the user input will be, the code outcome might not always be what the programmer originally intended.

The issue I’m going to talk about today is an issue that arose with people upgrading from DC++ 0.674 to 0.691. (I don’t know if the problem was or not possible with 0.674.)

A bug report and patch was swiftly created and will most likely be introduced in “0.692″ (upcoming version).

If you look at the patch, you will see that lines starting with ‘-’ is removed and ‘+’ is added. With this knowledge, the only “real” change you will see in the patch is that instead of “short” it says “unsigned short”. What this change really does lies in the way C++ (DC++ is written in C++) is designed.
(If you have programming knowledge, you know a short is a type for variables/functions etc. For the no-programming-knowledge-people; Consider that you have a box. This box can store a value, but only one, at any given time. Now, this box have to have a ‘type’, meaning it can only contain values of that certain type. short is one of those types.)

In C++, if a type has nothing before it, it is ’signed’; This means that it can contain values up to 2^15. The problem is that socket ports can reach 2^16. And this is where the ‘unsigned’ comes in the picture. If a type has ‘unsigned’ before, it not only support 2^15, but also 2^16. We come to the conclusion that using ‘unsigned’ before removes our issue.

Another thing you might notice with the current behaviour, it is that if you input a value above 2^15 (but below 2^16), you will then use - 2^16, this of course leading to a negative value. An interesting thing is that if you input something above 2^16, the value used will then also be - 2^16.

ADC!

Thursday, June 8th, 2006

You’ve probably seen a lot of references to ADC in the changelog for DC++, in the forum (people saying “no, this won’t come in NMDC, but in ADC” etc), the bugzilla, here and on the public DC developers hub.

What is ADC exactly then? It’s basically a way of defining how two clients (or client and server [hub]) communicate over the Internet. ADC is a protocol, just like HTTP (which you’ve probably noticed when you’re viewing web sites), FTP, IRC or NMDC. Think of this communication, the protocol, as two people in real life talking to each other. They both need to understand each other, otherwise they are just making noise with their mouths. Think of protocols as natural languages; English, French, Italian or Swedish.

ADC is not complete, but the current draft is what (basically) is going to be the final version.

What does this mean then? Well, it means different things, depending on who you are and what you want do.

For the common man, this is what ADC will bring (though, not necessarily will be in DC++);

  • Secure overall communication. This means the use of SSL (secure socket layer).
  • Secure password. This means it is impossible(1) for other people to find out what your password is when you’re logging into a hub. (Not depending on above.)
  • When searching, you can use regular expressions to accurately find what you are looking for
  • You can change nick when you are still connected to the hub (provided the hub allow you to).
  • Due to the identification scheme in ADC, different shares in hubs is possible.
  • Using partial file listing.
  • All text (between you and other users) are in UTF-8. This means that Chinese people can write characters and everyone sees them properly. Basically, any type of letter/symbol in any language will be viewable.
  • A native support for hashes (every client has to serve hashes).
  • If the hub chooses to, it can avoid sending main chat messages if you don’t want them.
  • Native support for “/me” (”* ullner is now away”). (That is, speaking in third person.)
  • Responses to searches can now be directed properly.

Wow. That’s a lot, ain’t it? And this is only what I can think of now. Think about when the protocol is finished!

So, what does ADC mean for developers?

  • A clear structure over what and when to send things. (There is no “spec” on how or when to send things in NMDC, only an semi-accurate order.)
  • We don’t need to have commands that are badly spelled ($LogedIn etc).
  • $ and | etc isn’t forbidden in commands.
  • Hub developers don’t need to depend on client developers to implement something for clients to safely log in.
  • One can change state and add features while connected.
  • Information about the client is now incremental, meaning you only send those parts that are changed.
  • Clients now don’t have to “guess” if two users in two hubs are the same.

So, I hope you have a better picture of what ADC is and what it can potentially bring.
(1) Unless the Tiger algorithm is broken, which is not likely in the next years.

Filtering Redux

Sunday, June 4th, 2006

In a past entry, Fredrik explained what caused the “Filtered” count in the Search Window. What is more interesting to me is the why. Why do all the search windows have to get all of the search results? It has to do with the design of the NMDC protocol by Jon Hess. The $Search sends out the essential information: search terms, size limitations, and search type. The $SR (search result) likewise sends the essential information: result type, directory name, file name, size. What is missing is a way to link a given $SR to the $Search that spawned it.

So, what happens when a user does two searches? The original NMDC client would not let the user do this. It didn’t have the ability to support more than one source for a given queued download, so there were no background searches for alternates. It also did not let the user open more than one window. DC++ does both, so it can have multiple searches happening simultaneously. To make sure each place gets its search results, it broadcasts them to all open search windows as well as the download queue. The windows then display the results that match their search criteria. This is the reason that ADC’s search mechanism has a token associated with it. Each search will have a different token, and each window will pick the results matching that token, with no guessing involved.

The limitations on the NMDC search also extend to other DC clients. Clients wishing to return hits based on the contents of the files will be filtered by DC++, since its search windows expect matches in the file or directory names.

Downloads and uploads

Sunday, May 7th, 2006

Ever wondered how much you have downloaded and uploaded? Well, here’s how to find out.

If you look at the status bar, you see (the first pair of) “D: “and “U: “. That is how much you have Downloaded and Uploaded in this session from when you started DC++. When you restart DC++, these two will be reset to 0.

The information in the status bar is added to the total amount of downloaded/uploaded, which you can find under Help -> About DC++ and at the bottom. This information will not go away when you close DC++. It will only go away if you delete dcplusplus.xml (or specifically if you delete the entries in it; <totalupload type=”int64″> and <totaldownload type=”int64″>).

Note that none of the above information is transmitted to other people.