Site/Off-Topic Discussion Thread 3

Electric Boogaloo

Quote# 123678

(Note: This thread's locked now 'cause the party has levelled up now!)

Let's talk about the site and other things!

Post feedback, issues, random comments, death threats, etc. here like in the other Site/Off-Topic discussion threads.

Site/Off-Topic Discussion Thread 3, Brought to you by Mookitty 130 Comments [1/16/2017 8:31:37 AM]
Fundie Index: 0
Submitted By: shy

1 2 3 4 5 6 | bottom


@dxdydz: The slow searches are partially a result of the poor database design, partially the result of the server, and partially the result of not having full-text search indexes on the fields you can search. That last one would make things a lot snappier but would require several gigs of database space, and my hosting plan only provides 5. I've toyed with hacking together manual hash tables to speed things up. I actually already have one on the Board field because it was inexplicably slow even compared to current search speeds. (If you try it, you'll see it's a bit faster than searching the other fields. Odd given I'm just using a basic modulo hashing function.)

Edit: It actually seems the Fundie and Quote Author fields have caught up a bit since I just rebuilt the indexes manually. Searching the Boards field is still a good 2-3 seconds faster on my end compared to searching them, though. The others are still way slower. I also just freed up enough space to re-enable the "mega-indexes" I made on the quotes and comment tables. Loading both quote and archive pages should now be faster, quite noticeably so. It definitely is on my end. If you guys are reading this, can you notice the difference?

1/19/2017 9:33:26 AM



The wait for a search to be completed is so long that even with shaving off 2 or 3 seconds it's hard for me to tell if there is a difference.

1/19/2017 12:23:55 PM


Random thought: the only thing that would make Lisp more obnoxious would be if it used Reverse Polish notation and was like Subject-Object-Verb language instead of Verb-Subject-Object one. Japanese people might like it, though.

1/19/2017 4:16:46 PM



Quoted fundie and member of fschit's Coalfart forum Cornfed (a.k.a. Cornholed) has infected FSTDT.

...and so has Tadpole:

Get a load of what he says at the end.

As per Col. Jenkins (Neil Patrick Harris) at the end of "Starship Troopers", 'It's afraid. It's afraid!'

You've gotten under his warty skin, shy.

1/19/2017 10:44:54 PM

Pharaoh Bastethotep

@Pepe post I just deleted:
Now who has proof Carib (sic; Caamib) isn't one of your own pet strawman? Anyone? You know this is ghostwritten by one of your eurocucks.


1/20/2017 1:45:09 AM


@Moose-Kun: Since this appears to be Cornfed's first post, I'm going to let him and his eloquent, thoughtful contribution stay. For now. I'm going to do a three strikes deal from now on so no one can whine about this place deleting dissenting opinions, banning dissenters, or becoming an echo chamber. I'm gonna be generous and not give him a strike for that post since it wasn't too bad (or maybe Herra Hullu is desensitizing me). But I'm in an unusually good mood right now, so if that changes I may also change my mind on letting him off the hook this time.

As for tadpole, we are going to start deleting everything from his posts, replacing it a message and the IP address, and then banning said IP. See his little 'present' you showed me for an example. (Follow my example, my modly minions.) I want to keep some evidence for when I report his harassment to AOL and Verizon. Even if they're not his real IPs, operating proxy servers or relay nodes is against both of their terms of service.

@Pharoah: lmao 'eurocucks'... fucking buzzword salad approaching postmodernist incoherence

1/20/2017 4:34:46 AM



Tadpole doesn't get the hint, does he...?:


He's just cruisin' for that TOS report to his ISP isn't he...?!

1/20/2017 8:27:02 AM

Pharaoh Bastethotep

Why am I getting an error message when trying to access the start site?

1/20/2017 3:13:30 PM


@Pharoah: Are you still getting it?

1/21/2017 5:29:21 AM


Yesterday at work, at noon, I wanted access the site for a short look, but got the database error. That was during the inauguration of our new Maximo Lider. My first thought was: "oh, the purge has started already". ;-)

1/21/2017 7:34:46 AM


Maybe you could store the authors in a separate table from the quotes, and refer to the authors everywhere else by surrogate key? There would be a unique constraint on author names in the authors table, so that the same author wouldn't get multiple author IDs.

That means you have two queries to search-by-author. The first one gets the author ID by searching the (considerably smaller) authors table. The second one searches the quotes table, but that involves walking a non-unique integer index, which is faster than scanning the entire table like we do now.

Besides being faster, storing author names in a separate table should also cut down on storage consumption, because we have a large number of quotes from the same author.

1/21/2017 10:01:08 AM


Seems quiet now... did it work? *blinks*

@pyro: FUCKING THANK YOU. If I had been the one who designed the database, that is exactly what I would have done. But when I mentioned this to a couple of other geeky friends, they more or less thought I was mad and it would dramatically increase latency and the amount of work the database had to do. Apparently indexes, cached queries, and strategic programmatically-maintained redundancy are things I hallucinated. That would dramatically increase search speed on those little fields. The overall performance elsewhere would probably be about the same. There'd definitely be a mild performance hit on INSERTs, but that's insignificant in the bigger picture given how disproportionate the read-to-write ratio is.

Implementing it now would take a lot of work because of having to clean out dupes, go back and manually fix quotes, etc. In this case, it was easier to hack a DIY hash table together in a separate table. I also figured out what was going on yesterday with the Board field anyway: SELECTing unnecessary rows and wrapping shit in transactions. Why would you even do that here?? I don't know WTF is up with this database, but it seems the more I check it out instead of just running basic queries, the more inexplicably bizarre shit I find.

BTW, the DBA Stack Exchange is a fucking trainwreck and I've found myself reading through it occasionally for a laugh.

1/21/2017 2:27:35 PM



I don't know WTF is up with this database

If whatever you've done causes a certain Turkish Tadpole to go 'WTF?!' when he tries to re-infect FSTDT again, then we'll know you've done a good job. b^_^d

Why would you even do that here?? indeed, Ugly Toad? I bet so many admins in other forums/boards have said the same about him.

Then he wonders why he experiences the refreshing taste of Mjolnir?

Would love to be a fly on his wall when he tries again.

1/21/2017 5:02:25 PM

Pharaoh Bastethotep

@Relevant part of a Frogbrain post in the new features thread I deleted:
If you had any brains, you might have realized I'm single handedly using a cheap monthly subscribtion VPN and definitely not even Turk-Telekom, only when I log on from different computers and smartphones on my way. When you work for a good corporation, they don't use retarded local internet, but satellite.

1/22/2017 5:48:18 AM


I ain't the fool here. I already knew he'd make one eventually, just not how long it'd take. He apparently doesn't realize that the idea is with each post he makes, he will be making fewer and fewer after that. I'm going to flat out admit the whole thing's essentially a naive Bayesian filter. I'm impressed the learning material I precalibrated it with seems to have blocked him for a whole 36 hours. We usually hear from him daily.

Another thing: Is his ingenious fool-proof VPN setup wrong? He said he didn't use Turk Telekom, but half of his comments originate from there. Also seems like an awfully lot of money and trouble to spend just to annoy people on the Internet. He seems a little agitated too. Could have taken a while for him to get one to slip through.

1/22/2017 8:43:04 AM


You know how cartoon supervillains have this fatal habit of bragging about their plans to the heroes, thus allowing our protagonists to find gaps in the evil scheme they would never have discovered if the baddie could just keep his trap shut? That's what the Amphibian's recent behaviour reminds me of.

1/22/2017 9:01:51 AM


@Uilleam: And that's exactly why I'm being somewhat vague on details when I say things about this matter. My being cryptic is not histrionic behavior.

1/22/2017 9:34:33 AM



Since this appears to be Cornfed's first post, I'm going to let him and his eloquent, thoughtful contribution stay

Just want to let you know how 'thoughtful' Cornholed can be. He and He Who Must Not Be Named? Not seeing any difference between them, considering the above.

And it gets worse...:

Q.E. and D.

No doubt he would use the same 'Context' excuse as the Croat Scroat.


Welp, there's definitely one less avenue for Croatia's worst lifeform to spew his BS.

Expect TimeToTurn - a.k.a. Kiwi Farms' resident shitstirrer Dynastia - to return, then. If he doesn't get trolled to death if he tries to migrate to ED Forums (They've created a post-elimination thread section. Some Farmers have found... 'shelter' there)

This is an ex-Kiwi. It has ceased to be.

1/23/2017 4:12:14 PM



Good riddance to bad rubbish in both cases. Love how they both went down in an explosive burst of karma, too.

1/23/2017 4:55:13 PM


I made some errors in my last submissions to CSTDT, sorry about that. But I've included the fixes in comments on the quotes. Could someone edit the fixes in?

1/24/2017 12:10:34 PM


@dxdydz: I'll get 'em in the morning. I'm about to go to bed. Always check this thread before I do.

1/24/2017 8:08:28 PM


Hip hip hooray, the Amphibian's back. More of the usual shit from him, but I recommend tightening the troll filter up as hard as possible so his "friends" can't raid us.

1/27/2017 8:38:42 AM



The Ugly Toad's back:

Time to test your new weapon. >:D


Blast! Ninja'd by Uilleam. But good work, though. And as the emergency services say, better to get several calls about the same accident/fire etc than none at all.

...oh, and another troll alert:

Nameless Coward under another screename/IP? Or a noisome little brat in need of an education?

1/27/2017 9:31:46 AM


@Moose-Kun: I knew a few of his posts would manage to sneak through every now and then. That's just the nature of a Bayesian filter. Notice how the text of these don't contain too much of his typical textual diarrhea. The IP's first 8 bits also don't resemble any of his previous with any statistical significance because it is in a rather 'dense' range of them, so it evaded statistical detection, which only checks the first eight bits. Once the last byte of an IP has > 5 trolls in it, a ban is done, meaning all IPs in the range [ ..] are banned. (Notice how I don't use the 9th bit) I don't check the username field because letting 'em identify themselves ultimately works against them.

I admit I've got the threshold too low if "cucks" slips through. Surprisingly, it turns out "cucks" is actually a hapax legomenon, this being the first time it's been collected in training data. It's the singular "cuck" that has 4 occurrences, one marked troll, the other three legitimate users quoting fundies who used the word, giving it a a troll:total ratio of 0.25 and nontroll:total ratio of 0.75. But if we used the entire database as training data (not feasible due to the amount of resources it would consume), the scores would be closer to 0.5 based on my general impression. That's why until you have a good-sized word corpus to work with, it's good to artificially deflate the significance of correlation, e.g. by taking the mean of each number and 1/2 so the scores tend closer toward 1/2 and closer resemble likely future scores, which tend toward 0.5 with a few notable exceptions.

Bayesian filtering might eventually force Pepe to start making intelligent posts to get through. By that point, the denominators of successive compounding in the logarithmic domain for certain words will approach 0 (troll probability 0) or infinity (troll probability 1), causing a division by zero error, or its lesser known correlate, a division by infinity error. Both are undefined.

After two weeks, instead of strengthening the filter, I am going to turn it off to get an idea of how effective it has been before adjusting anything. When I'm confident of its effectiveness, I'll probably release most of the source code, since that will mean any outstanding bugs have been ironed out, and the whole world has to answer right now WHO'S BAD?! seeing how it works won't have much effect on its effectiveness. I have to confess I totally started singing MJ's Bad at my computer screen when Pepe made his very first post after I enabled my toy cuz srsly WHO'S BAD?!? Pepe wasn't able to get a post through for six days. CHAMON!

1/27/2017 12:40:06 PM

1 2 3 4 5 6 | top: comments page