Powered by glolg
Display Preferences Most Recent Entries Chatterbox Blog Links Site Statistics Category Tags About Me, Myself and Gilbert XML RSS Feed
Saturday, Mar 01, 2014 - 16:09 SGT
Posted By: Gilbert

The March Of Ideas

An empty corridor in a deserted building. A lone, naked lightbulb swung forlornly on bare wires, painting it a cheap kaleidoscope of dim light and hesitant shadow. The dead air hung heavily, uncomfortably, for what seemed to be ages, before a nearly imperceptible breeze wound its way along the floorboards. A tiny figure sidled slowly from a dark corner.

Trenchcoat-clad figure: *in a raspy voice* I have got that which you want. Have you got that which I want?

Me: Maybe. Let's see the goods.

*Trenchcoat-clad figure proffers a ratty briefcase*

Me: *flipping through contents* Yes, this looks quite satisfactory indeed. Good job.

Trenchcoat-clad figure: Erm, can I have the briefcase back, please? It's the only one that I have.

Me: *sighs* Sure thing, Mr. Robo, "codename Robospierre". And may I ask, what are you doing, skulking about dressed like a supporting character in a budget noir flick?

Mr. Robo: *hurt* I paid good money for this getup!

Me: On the other hand, it's pretty spiffy.

Mr. Robo: *mollified* Thank you.

Me: But, I mean, you could have just emailed these documents to me, instead of going to these lengths.

Mr. Robo: *nervously* Can't be too careful. This was partly done during work hours, and Mr. Ham may be monitoring all electronic correspondence. It was only by a great stroke of luck that I came across a very nice hamster loitering outside the lab, who taught me this method of smuggling data out, and went as far as to sell me this trenchcoat at a discount. Got to be properly attired for such deeds, you know.

Me: Well, fine, but if I may offer a suggestion, the... headlamp doesn't really go with the mood.

Mr. Robo: How else would I find my way? It's dark in here!

Me: Err, ok. Well done, on the research at least. Here's your twenty bucks, as agreed, with a ten-dollar tip on top of that for being on time. Just curious, how much did the trenchcoat cost?

Mr. Robo: Five hundred only. I told you, he was very nice.

Me: *checks on smartphone* Actually, there are plenty of similar ones going for fifty or less on eBay. Wait a minute, what did that nice hamster look like?

Mr. Robo: I'm not sure, he was wearing a ski mask...

Me: Are you alright? Why are you staring at the ceiling like you've seen a ghost? Wait, don't run!

*turns around*


*Drat, discovered!*
(Original source: flickr.com)


Me: You.

Mr. Ham: Assuredly, Sir Ham H. Let at your service.

Me: And what business do you have here?

Mr. Ham: Oh, just tailing my loyal employee, Mr. Robo. Wasn't too hard, what with the experience I have shaking off private investigators. He was so anxious to complete our trenchcoat transaction that he paid a hundred extra, which I was about to return to him, but now he's gone and disappeared. What a shame. So, what extracurricular activities has he been up to? Also, I'm disappointed that you paid him just S$30.

Me: Hey, I'm not a government funding agency with money coming out of my nose. But you don't seem too angry.

Mr. Ham: No, not at all. You can't really expect employees to keep twenty-hour days, I was just joking with him. In any case, he was quite diligent in playtesting FappyBird - wait, that sounds slightly wrong - these few days, so I'm hardly against him having a personal project or two on company time. Unless it's commercialisable, in which case I want my usual cut.

Me: Oh, you don't need to be concerned in that case. I thought that I had observed some pattern in the references of papers that I had been reading in the course of my work, and being kind of busy outsourced the quantitative analysis to Mr. Robo. Tentatively, my suspicions are confirmed - from his technical report, "A Citation Source Study In Artificial Intelligence", simply by studying the references of a paper and completely ignoring its actual content, it can be determined to belong to a first-tier or third-tier conference with nearly 85% accuracy.

Basically, for each paper, its references were extracted, and those citing a recognized computer science conference had the generally-accepted tier of that conference put under consideration. Despite only about 40% of references being from such conferences, there is a big distinction in distribution between top conferences (in red) and also-rans (in blue):



As can be seen, top conferences cite other top conferences some 60% of the time, and not-so-selective conferences rarely, while less-selective conferences are far more indiscriminate. Further, this effect is remarkably consistent, notwithstanding the varying cultures of the conferences.

Interestingly, it also appears that it doesn't matter that much if the reference is from the same conference as the paper it is submitted to, just that it is in the right tier - for example, both AAAI and NIPS cited ICML more than themselves in 2013, normalised for total number of previous papers available.

Of course, as stated in the conclusions, simply cramming top reference sources into a paper probably won't save it if it's simply lousy, but this does suggest that including too many references from lesser-known venues might negatively impact the chance of a paper getting accepted at a good conference. However, this may also imply that ideas may get discounted or lost if they appear in not-so-prestigious collections, unless they're truly earth-shattering.

Mr. Ham: Isn't there a saying, "from good shall come good, and from evil only evil?"

Me: That's overly dramatic, but given that promising classifications were possible with such limited data, one can only wonder at how capable AI reviewer agents would be if supplied with far more information, in particular the profile of past rejected papers. Given that all indications are that the acceptance decisions of separate program committees are not particularly congurent (about 41%, though with only ~15% accept rate), it could be amusing - and perhaps a little unsettling - if a team of bots could achieve comparable inter-rater agreement, though without providing comments.

That said, if the bot creators are not careful, they may end up fixating on a not-very-relevant attribute, such as word count - which is probably far less of a concern in conference settings due to page limits - but I think we've probably gone past that. Question then would be if any adventurous AI conference might want to to test the hypothesis...

*sighs* Trouble is, one gets the sense that there is no time, and even less of it as it goes by. So many possibilities to flesh out - I should really get around to writing up all the findings that I've gotten these past years in proper form - some of those even presented on this blog.

Mr. Ham: As I told you, use LaTeX and sprinkle in references liberally, and your credibility instantly doubles, eh? But one wonders why anybody would bother, given the rewards.

Me: I don't know, just last week, one of my former colleagues revealed on Facebook that one of his papers in a reputable IEEE journal had been ripped off and offered for sale for 4500 Rupees (~S$90), supposedly with accompanying code - which if true, is quite the bargain. Heck, the sellers even produced promotional videos, and a badly-mangled version of the paper has already made it into at least one other journal! This trend is only going to get more delicate to deal with... and no, Mr. Ham, don't even think about it. And stop taking notes.

I can only admire those pure mathematicians, whom devote their lives to truths of no apparent application, but whom I have neither skill nor inclination to emulate. Haven't even found the time to get into the book that I just received as a token for getting my essay selected in a competition on the relevance of the Solidarity movement organized by the University of Warsaw, either.

Mr. Ham: Couldn't resist dropping that bit of info, could you?

Me: *shrugs* I'll take what pep-ups I can right now. Though, quite earnestly, the incumbents could do to move a bit more left, this being a large part of why I prepared the essay in the first place. We'll see how they do.

Mr. Ham: It's still not looking good, I'm afraid - they aren't even getting arrested over the right sort of tarts. And, uh, one last thing before you leave.

Me: Yes?

Mr. Ham: Could you extricate me from this hole? I'm kinda stuck.



comments (0) - email - share - print - direct link
trackbacks (2) - trackback url


Next: A Post In Three Parts


Related Posts:
Think Thunk
Groundwork
The Week After
It Was A Honest Mistake
The Huntster Awakens
Back to top




2 trackbacks

Trackback by Isabel Marant Dicker Boots Look Alike
Isabel Marant Dicker Boots Look Alike - [bert's blog]


August 16, 2014 - 08:34 SGT     
Trackback by Vietnam Visa from Hong kong
Vietnam Visa from Hong kong - [bert's blog]


September 23, 2014 - 19:04 SGT     

Copyright © 2006-2025 GLYS. All Rights Reserved.