[00:14:50] *** EvolutionCrazy has quit IRC
[00:24:33] *** Taube is now known as taube
[00:49:28] *** rcjsuen has joined #bittorrent
[01:02:40] *** [diablo] has quit IRC
[01:22:26] *** Gregory_Lion has joined #bittorrent
[01:22:36] *** Gregory_Lionhear has quit IRC
[01:36:33] *** lioux has quit IRC
[01:37:00] *** lioux has joined #bittorrent
[02:03:09] *** lioux has quit IRC
[02:04:18] *** firefly2442 has quit IRC
[02:37:04] *** The_8472 has quit IRC
[03:11:55] *** pitoow has joined #bittorrent
[03:12:50] *** pitoow has quit IRC
[03:13:37] *** pitoow has joined #bittorrent
[03:38:52] *** slipstream-- has joined #bittorrent
[03:51:24] *** slipstream has quit IRC
[03:53:17] *** lioux has joined #bittorrent
[04:07:52] *** lymeca_ is now known as lymeca
[05:13:26] *** [diablo] has joined #bittorrent
[05:25:31] *** cosmodad has quit IRC
[06:05:33] *** rcjsuen has quit IRC
[07:35:19] *** adante has quit IRC
[07:39:39] *** adante has joined #bittorrent
[07:41:24] *** MassaRoddel has joined #bittorrent
[07:46:34] <TheSHAD0W> http://cgi.fark.com/cgi/fark/youtube.pl?IDLink=3326011
[07:55:22] <thewird> haha
[07:55:25] <thewird> break the DMCA
[08:07:04] *** init0_ has joined #bittorrent
[08:19:31] *** init0 has quit IRC
[08:22:34] *** danggun has joined #bittorrent
[08:23:17] <danggun> I'm experiencing a strange problem when I'm transferring bittorrent traffic. My http requests get interrupted and/or do not make connections at all.
[08:23:36] <danggun> When visiting a webpage I have to repeatedly refresh the page until the http connections are established and content begins downloading.
[08:24:09] <danggun> My connection is not being overloaded as I've throttled my client all the way down to 10kB up/down. But when I close the bittorrent program the problem immediately goes away.
[08:26:11] <uau> how many peers/seeds are you connected to?
[08:27:10] <uau> could be a problem with a connection number limit somewhere; some routers suffer from that for example
[08:27:36] <danggun> ahh i see. how can i check that? would that be the total number of nodes?
[08:30:15] <uau> your client should show how many peers/seeds you're connected to
[08:31:00] <uau> if you have a client with DHT support enabled that can also affect some routers (if they try to keep track of every other machine you're exchanging UDP packets with)
[08:31:27] <danggun> i'm using utorrent and i can see the number of peers/seeds per torrent but i don't know how to get an aggregate count.
[08:31:55] <uau> you have several torrents running simultaneously?
[08:32:08] <danggun> yes
[08:32:21] <uau> you shouldn't if you have it throttled to 10 kB/s
[08:32:42] <danggun> i don't. i'm just saying that i tested it at that low speed and still had the problem.
[08:32:51] <danggun> usually i'm uploading at 35kB and downloading up to 200kB.
[08:33:00] <uau> how many torrents do you have running?
[08:33:13] <uau> 35 kB/s is still kind of low for multiple torrents at once
[08:33:54] <danggun> right now there are 16 active. all but one are uploading. and 80% of them have less than five peers/seeds each.
[08:34:24] <uau> how many peers do the others have?
[08:34:46] <danggun> 17, 34, 100, 194
[08:35:07] <uau> that's over 300
[08:35:12] <danggun> but actually, that's the bigger number of the two.
[08:35:14] <danggun> do you use utorrent?
[08:35:58] <uau> (just to make sure, is the the number you're connected to or some total in torrent from scrape or such?)
[08:36:01] <uau> no i don't use utorrent
[08:36:05] <danggun> so for example... the most active torrent says this:  Seeds: 0 (101), Peers 9 (194)
[08:36:20] <danggun> so does that mean i'm connected to 9 peers or to 194?
[08:36:22] <uau> are you seeding that one?
[08:36:27] <danggun> yes
[08:36:36] <uau> most likely means you're connected to 9
[08:36:58] <uau> that 9/194 is rather low
[08:37:11] <uau> i'd expect you to get more connections if there are that many peers
[08:37:12] <danggun> in that case, all 16 torrents are "connected" to less than 10 peers each.
[08:37:31] <uau> hmm that's somewhat weird
[08:37:51] <danggun> well, the torrents i share are not very popular so that's why they have such low numbers.
[08:38:16] <uau> well the output did claim there were a total of 194 peers
[08:38:46] *** camrdale has quit IRC
[08:39:05] <danggun> ok well i'll try pausing all the torrents except for one and see what happens.
[08:39:58] <uau> utorrent probably has DHT active; you could also test whether disabling that affects things
[08:40:27] <danggun> ok
[08:50:05] <danggun> hmm... it's hard to tell if it's working. it seems a bit better but even with only 3 torrents and dht turned off i still get timeouts.
[08:50:27] *** [diablo] has quit IRC
[08:59:02] *** danggun has quit IRC
[09:10:25] *** [diablo] has joined #bittorrent
[09:17:46] *** westside has quit IRC
[10:14:31] *** init0_ is now known as init0
[10:14:40] <init0> moin
[10:54:07] *** EvolutionCrazy has joined #bittorrent
[11:12:01] *** The_8472 has joined #bittorrent
[11:36:56] *** lioux has quit IRC
[11:37:51] *** lioux has joined #bittorrent
[13:43:27] *** rcjsuen has joined #bittorrent
[13:48:02] *** The_8472 has quit IRC
[13:48:26] *** The_8472 has joined #bittorrent
[14:18:01] *** funkywizard has quit IRC
[14:18:09] *** funkywizard has joined #bittorrent
[14:42:45] *** s_20 has joined #bittorrent
[14:43:08] *** s_20 has left #bittorrent
[14:47:03] *** cosmodad has joined #bittorrent
[14:55:14] *** [diablo] has quit IRC
[15:02:00] *** Gregory_Lion has left #bittorrent
[15:06:50] *** Elperion has joined #bittorrent
[15:19:39] *** Elperion has quit IRC
[15:19:41] *** lioux has quit IRC
[15:19:45] *** danomac has quit IRC
[15:19:47] *** Mitchman- has quit IRC
[15:19:49] *** stray has quit IRC
[15:19:50] *** Nolar has quit IRC
[15:20:36] *** Elperion has joined #bittorrent
[15:20:36] *** lioux has joined #bittorrent
[15:20:36] *** danomac has joined #bittorrent
[15:20:36] *** Mitchman- has joined #bittorrent
[15:20:36] *** stray has joined #bittorrent
[15:20:36] *** Nolar has joined #bittorrent
[15:23:30] *** rcjsuen has quit IRC
[15:25:30] *** Andrius has joined #bittorrent
[15:36:21] *** norc has joined #bittorrent
[15:39:25] <norc> Are there any bittorrent trackers available that use procedural languages for announces?
[15:39:35] <norc> (Just talking about software)
[15:39:54] <norc> I want to know if I'm the first to use them or not. ;)
[15:40:46] <TheSHAD0W> You mean like PHP?
[15:40:53] <norc> No, like PL/pgSQL
[15:41:03] <norc> Or plsql in oracle
[15:41:51] <norc> Because I find it hard to imagine that I'm the first to be able to handle hundreds of announces per second, just because I use it properly.
[15:43:14] <norc> All the major softwares like XBT, BNBT, etc use MySQL with tons of seperate queries, updates and inserts
[15:43:53] <TheSHAD0W> How many peers does that translate to?
[15:44:27] <norc> Well, my test cases show that an announce is processed completely within 8ms. Testdata were about 100.000 peers perhabs
[15:44:46] <norc> on 500.000 torrents
[15:45:07] <norc> (I created the testdata that they would match a realworld example)
[15:46:43] <norc> its said to see sites to rent more than ten servers because their software cant handle the amount of requests hammering their site
[15:46:50] <norc> s/said/sad/
[15:47:40] <norc> instead of using the database which is already able to optimize more than anyone the developers start tweaking on their dispatcher
[15:47:50] <norc> but using some fcgi script would be enough to handle that
[15:48:07] <norc> Just create the right relations
[15:48:10] <norc> and its done ;)
[15:50:59] * ShadowJK thought the fast trackers did not use *SQL at all :-)
[16:02:08] *** stiv2k has left #bittorrent
[16:08:42] *** rcjsuen has joined #bittorrent
[16:19:38] *** The_8472 has quit IRC
[16:25:12] *** The_8472 has joined #bittorrent
[16:33:28] <DreadWingKnight> BNBT doesn't require you to use mysql
[17:18:12] *** rcjsuen has quit IRC
[17:24:11] *** cyb2063 has joined #bittorrent
[17:39:08] *** GoussX has quit IRC
[17:39:49] *** GoussX has joined #bittorrent
[17:44:02] *** EvolutionCrazy has quit IRC
[17:51:24] *** Andrius[] has joined #bittorrent
[17:51:45] *** Andrius has quit IRC
[17:52:30] *** Andrius[] has quit IRC
[17:52:59] *** Andrius has joined #bittorrent
[18:04:00] *** GoussX has quit IRC
[18:04:01] *** GoussXX has joined #bittorrent
[18:46:27] *** rcjsuen has joined #bittorrent
[19:01:50] *** EvolutionCrazy has joined #bittorrent
[19:13:44] *** [diablo] has joined #bittorrent
[19:15:57] *** rcjsuen has quit IRC
[19:20:56] <norc> DreadWingKnight: even so
[19:21:06] <norc> just using another rdbms wont speed up things
[19:21:26] <norc> ShadowJK: I dont think you can accomplish the same thing without a database at reasonable speeds
[19:21:44] <norc> a database is specifically designed to work with extremely large amounts of data very fast
[19:22:13] <norc> DreadWingKnight: its the pl/pgsql (or pl/sql) Im after
[19:24:47] <The_8472> i think he meant you could use a scratch file
[19:28:45] <ShadowJK> tpb uses opentracker, right...
[19:29:00] <norc> The_8472: I doubt that would do any good
[19:29:33] <The_8472> ShadowJK, they do
[19:29:48] <The_8472> http://opentracker.blog.h3q.com/mrtg/
[19:30:06] <ShadowJK> Is there any bigger benchmark than tpb? :)
[19:30:44] <The_8472> not on this planet
[19:31:01] <ShadowJK> Right, so they're doing 10,000 requests per second per tracker and almost 5 million users per tracker...
[19:31:36] <The_8472> that TCP stack probably required some heavy tuning ^^
[19:31:50] <norc> ShadowJK: That's not a problem, if you have a multi core system with a shitload of ram. :)
[19:32:02] <norc> besides, mysql is the fastest rdbms, no doubt about that
[19:32:14] <norc> but you dont have transactions
[19:32:19] <norc> ACID is not available.
[19:32:20] <The_8472> you do
[19:32:29] <The_8472> you just need the InnoDB engine
[19:32:34] <The_8472> that one does ACID
[19:32:59] <The_8472> and they're working on a new one, "falcon" iirc
[19:33:09] <ShadowJK> neither opentracker nor hypercube use *sql, right?
[19:33:15] <The_8472> correct
[19:33:35] <The_8472> they use in memory storage i think
[19:33:42] <The_8472> or maybe a file
[19:33:57] <ShadowJK> opentracker doesn't write anything to HD it'd seem
[19:34:27] <norc> The_8472: Well, InnoDB. I could start ripping it apart. ;)
[19:34:42] <norc> I know why I'm working with Postgres, and I'm happy with it.
[19:35:40] <The_8472> InnoDB is fine if transaction safety and constraints (i.e. proper DB layout) are more important than raw throughput
[19:37:13] <The_8472> falcon engine is supposed to do both... concurrency and transaction safety
[19:37:50] <norc> Still, PostgreSQL is ages ahead of MySQL.
[19:37:51] <norc> :)
[19:38:21] <norc> InnoDB might be fine, but not enough for my applications.
[19:39:20] <ShadowJK> I was using ~46 bytes ram per peer in tracker I once tried to write... would have to slim that down alot for 5M peers in ram :)
[19:39:26] <ShadowJK> I guess..
[19:39:51] <norc> ShadowJK: Sweet. One crash and you have a problem. :)
[19:39:54] <The_8472> well, they probably have a few gigs of ram too
[19:39:55] <kjetilho> hmm?  that's just 230 MB
[19:40:02] <ShadowJK> norc, what do you mean :)
[19:40:05] <The_8472> norc, no... not really... peers will reannounce anyway
[19:40:09] <ShadowJK> Yeah
[19:40:20] <The_8472> and opentracker supports instance-syncing too
[19:40:31] <ShadowJK> Wait half an hour and everyone is back again
[19:40:39] <The_8472> so if one crashes there'll still be the other one(s)... just more work for the loadbalancer
[19:40:57] <The_8472> and then there's still PEX and DHT... a tracker being down is not really an issue
[19:41:35] <ShadowJK> Though if you're not allowing any random torrent, you'd probably want to save list of allowed torrents to disk
[19:41:55] <The_8472> that's why it's named "opentracker"
[19:42:01] <ShadowJK> :)
[19:42:52] *** camrdale has joined #bittorrent
[19:55:32] *** norc has quit IRC
[19:55:45] *** norc has joined #bittorrent
[19:56:08] <norc> The_8472: Perhabs I just like to follow conventions and design. :)
[19:56:23] <norc> And usually you solve such things with databases, or database clusters
[19:56:40] <norc> Of course you can keep them in memory and try to find them there
[19:56:46] <norc> but even then a database does about the same thing
[19:57:08] <norc> with the difference that it knows the statistics, can plan the query accordingly, use special indexes
[19:57:27] <The_8472> you can do the same with your custom in-memory storage
[19:57:38] <The_8472> hash tables, trees, statistics...
[19:57:54] <norc> The_8472: yeah, and the more you optimize the closer you get to a real database
[19:57:59] <norc> the more you do that stuff
[19:58:06] <The_8472> btw, a file system is a database too, even provides fine-grained locking (depending on the OS) ^^
[19:58:22] <ShadowJK> And the closer you optimize the database, the closer you get to a real program? ;)
[19:58:23] <norc> Well, but its a hierachial database.
[19:58:24] *** firefly2442 has joined #bittorrent
[19:58:25] <The_8472> no, a real database is general-purpose
[19:58:53] <norc> The_8472: generally the database is able to optimize all by itself
[19:58:55] <The_8472> using a highly-customized solution can shave off the necessary ram or cpu cycles to squeeze in a few 100k more peers
[19:59:00] <norc> if it cant do that you're not using a good database.
[19:59:26] <The_8472> especially if you have to use a TCP connection to talk to the database
[19:59:47] <norc> The_8472: Sure, if you want to create unmaintainable code, it's the right thing to do that. Write everything in ASM designed on the architecture of your server. Use every mathematical algorithm and idea you can think of
[20:00:10] <The_8472> you are exaggerating
[20:00:18] <The_8472> there is a middle path between both
[20:00:26] <norc> which is considerably small
[20:00:29] <The_8472> use customized data structures, write in C for example
[20:01:05] <The_8472> talking through some API / socket connection to a DB is slower than doing some pointer magic no matter what you do
[20:01:31] <norc> The_8472: You just send a few bytes, the name of a stored proc + some args
[20:01:57] <The_8472> which is already too much if you want to squeeze the max. out of the machines you have available
[20:02:07] <norc> The_8472: now thats exaggerating
[20:02:11] <The_8472> this won't be an issue if you have to serve 10k users
[20:02:19] <norc> 800 or 820 requests per second is not a difference
[20:02:19] <The_8472> but it is if you serve millions
[20:02:36] <norc> The_8472: If you serve millions you can get a database cluster instead
[20:02:46] <norc> and usually thats what you do if you have such a problem
[20:03:06] <The_8472> why would you need a database cluster? you'd still need the machines doing the webfrontend stuff
[20:03:21] <The_8472> if you use a cluster then just run a cluster of your customized software on it
[20:03:27] <The_8472> we're talking about trackers here
[20:03:36] <norc> The_8472: of course then you need to design new software to coordinate with this cluster
[20:03:54] <The_8472> it's a very primitive task... only throughput matters... not some fancy transaction guarantees... heck... even a lost insert once in a while won't matter
[20:04:13] <norc> right, whats some data loss, wont matter ;)
[20:04:15] <ShadowJK> Doing dozen arithmetic ops and one or two pointer dereferences is going to be a few orders of magnitude faster than the system call to send query to the database...
[20:04:48] <The_8472> if we were talking about clustering something like amazon then you're right... it's a complex system AND demanding throughput
[20:05:05] <The_8472> but if it's a primitive system demanding throughput then a custom solution can be much much faster
[20:05:06] <norc> Perhabs I'm too customized since Im used to high-demand environments
[20:05:22] <norc> In my opinion data loss is unacceptable, in any case
[20:05:52] <norc> The_8472: Alright, you said yourself, opentracker is very simple
[20:05:52] <The_8472> not in this particular case
[20:05:56] <norc> my tracker is way more complex ;)
[20:06:07] <The_8472> different requirements, different purpose
[20:06:09] <norc> yeah
[20:06:21] <norc> anyway, I gotta go get some food, I'll be back later
[20:06:22] <The_8472> remember that those things are run by non-commercial entities
[20:06:32] <The_8472> geek-power is free, hardware isn't
[20:08:08] <ShadowJK> Consider that bittorrent is very resilient, it has to deal with all the fuckups of ISPs, consumer routers and Internet in general..
[20:09:21] <ShadowJK> Heck, you don't even have to give the peer a "correct" answer, or even an "optimal" answer, it will work fine even if you forget to answer them half of the time and give them 50% false data :-)
[20:09:35] <The_8472> opentracker actually does that
[20:09:37] <The_8472> well... not 50%
[20:09:43] <The_8472> but it inserts a few fake IPs
[20:09:56] <ShadowJK> Well you'll end up giving false data anyway that  you aren't aware of
[20:10:07] <firefly2442> why would a tracker give a fake IP?
[20:11:01] <The_8472> http://opentracker.blog.h3q.com/?p=22
[20:12:33] <firefly2442> ahh interesting, thanks
[20:12:34] <ShadowJK> Hey I just thought of something
[20:15:17] <ShadowJK> Do you actually need to store all 100,000 peers of a torrent? :-)
[20:15:37] <ShadowJK> Surely you could get by if you store a random sample of the swarm ;-)
[20:16:13] <The_8472> well... sortof, but not all clients do PEX
[20:16:28] <The_8472> and then there is the issue with clients being firewalled / not connectable
[20:16:57] <The_8472> so it reduces clustering if you store all of them and return them at random (or rather... with a bit smartness... don't hand out seeds to other seeds)
[20:17:18] <ShadowJK> Yeah the clustering thing...
[20:18:41] * ShadowJK sleeps
[20:19:03] <ShadowJK> Was it bram who claimed PEX causes clustering
[20:19:12] <The_8472> i think so
[20:19:18] <The_8472> but that's not really true
[20:19:24] <The_8472> it actually can do the opposite ^^
[20:20:38] <ShadowJK> I guess this is where there approach differs actually, norc is thinking of it as engineering a database
[20:20:51] <ShadowJK> And then there's me who's thinking of a diverse swarm
[20:21:07] <ShadowJK> and the tracker as a tool to help create that swarm
[20:22:45] <The_8472> yeah, p2p systems tend to have some inherent chaos anyway... so you don't need something highly accurate on the tracker's side
[20:32:04] *** Elperion has quit IRC
[20:52:17] *** rcjsuen has joined #bittorrent
[21:28:49] *** GoussXX has quit IRC
[21:41:07] *** stray_ has joined #bittorrent
[21:47:16] <DeHackEd> well that was annoying. I lost X11
[21:53:01] *** GoussX has joined #bittorrent
[21:53:40] *** stray has quit IRC
[22:07:41] <thewird> that would be annoying
[22:08:24] <DeHackEd> yeah, really weird. strace said it was calling futex... which is a bit strange since X is single-threaded. I'm guessing the nvidia drivers hosed themselves up.
[22:30:38] *** jlouis_ has quit IRC
[22:31:08] *** jlouis has joined #bittorrent
[22:36:17] *** EvolutionCrazy has quit IRC
[22:36:20] *** jlouis has quit IRC
[22:39:16] *** jlouis has joined #bittorrent
[22:43:33] *** jlouis has quit IRC
[22:44:09] *** jlouis has joined #bittorrent
[22:51:31] *** EvolutionCrazy has joined #bittorrent
[22:56:12] *** EvolutionCrazy has quit IRC
[23:02:40] *** EvolutionCrazy has joined #bittorrent
[23:04:22] *** cyb2063 has quit IRC
[23:21:53] *** slipstream has joined #bittorrent
[23:33:14] *** maruchanil1 has joined #bittorrent
[23:33:28] <maruchanil1> hello
[23:33:41] *** slipstream-- has quit IRC
[23:33:59] <maruchanil1> does anyone have a waffles invite?
[23:34:44] <__henke__> no
[23:36:26] *** maruchanil1 has left #bittorrent
[23:36:26] *** The_8472 has quit IRC
[23:43:09] *** [diablo] has quit IRC
[23:45:16] *** MassaRoddel has quit IRC
[23:50:17] *** funkywizard has quit IRC
[23:50:29] *** funkywizard has joined #bittorrent

top