April 2006

The botnet hunters

From The Washington Post‘s “Bringing Botnets Out of the Shadows“:

Nicholas Albright’s first foray into some of the darkest alleys of the Internet came in November 2004, shortly after his father committed suicide. About a month following his father’s death, Albright discovered that online criminals had broken into his dad’s personal computer and programmed it to serve as part of a worldwide, distributed network for storing pirated software and movies. …

From that day forward, Albright poured all of his free time and pent-up anger over his father’s death into assembling “Shadowserver,” a group of individuals dedicated to battling large, remote-controlled herds of hacked personal PCs, also known as “botnets.” …

Each “bot” is a computer on which the controlling hacker has installed specialized software that allows him to commandeer many of its functions. Hackers use bots to further their online schemes or as collection points for users’ personal and financial information.

“I take my [handheld computer] everywhere so I can keep tabs on the botnets when I’m not at home,” Albright said …

On a Sunday afternoon in late February, Albright was lurking in an online channel that a bot herder uses to control a network of more than 1,400 hacked computers running Microsoft Windows software. The hacker controlling this botnet was seeding infected machines with “keyloggers,” …

Albright had already intercepted and dissected a copy of the computer worm that the attacker uses to seize control of computers — an operation that yielded the user name and password the hacker uses to run the control channel. By pretending to be just another freshly hacked bot reporting for duty, Albright passively monitors what the hackers are doing with their botnets and collects information that an Internet service provider would need to get the channel shut down.

Albright spied one infected PC reporting data about the online activities of its oblivious owner — from the detailed information flowing across the wire, it was clear that one of the infected computers belongs to a physician in Michigan.

“The botnet is running a keylogger, and I see patient data,” Albright said. …

“Anything you submit to law enforcement may help later if an investigation occurs,” he said. “Chances are, though, it will just be filed away in a database.”

Botnets are the workhorses of most online criminal enterprises today, allowing hackers to ply their trade anonymously — sending spam, sowing infected PCs with adware from companies that pay for each installation, or hosting fraudulent e-commerce and banking Web sites. …

… in the 13-month period ending in January, more than 13 million PCs around the world were infected with malicious code that turned them into bots.

… Shadowserver locates bot networks by deploying a series of “honeynets” — sensors that mimic computers with known security flaws — in an effort to lure attackers, allowing the group to capture samples of new bot programs. …

Shadowserver submits any new or undetected specimens to the major anti-virus companies. Andrews said he is constantly surprised by the sheer number of bot programs that do not get flagged as malicious by any of the programs. …

In Andrews’s experience, by far the most common reason criminals create botnets these days — other than perhaps to sell or rent them to other criminals — is to install online ad-serving software that earns the attacker a few pennies per install. …

Even after the Shadowserver crew has convinced an ISP to shut down a botmaster’s command-and-control channel, most of the bots will remain infected. Like lost sheep without a shepherd, the drones will continually try to reconnect to the hacker’s control server, unaware that it no longer exists. …

“Bot hunting can really take over your personal life, because to do this right you really have to stay on top of it — it can’t just be something you do on the weekends,” he said. “I guess it takes a special type of person to be able to sustain botnet hunting. … I don’t know anyone who pays people to do this kind of work.” …

Albright said that while federal law enforcement has recently made concerted efforts to reach out to groups like Shadowserver in hopes of building a more effective partnership, they don’t have the bodies, the technology, or the legal leeway to act directly on the information the groups provide. …

“Sadly, without more law enforcement support this will remain a chase-your-tail type game, because we won’t ever really shut these networks down until the bot master goes to jail, and his drones are cleaned.”

The botnet hunters Read More »

John the Ripper makes password cracking easy

From Federico Biancuzzi’s “John the Ripper 1.7, by Solar Designer“:

John the Ripper 1.7 also improves on the use of MMX on x86 and starts to use AltiVec on PowerPC processors when cracking DES-based hashes (that is, both Unix crypt(3) and Windows LM hashes). To my knowledge, John 1.7 (or rather, one of the development snapshots leading to this release) is the first program to cross the 1 million Unix crypts per second (c/s) boundary on a general-purpose CPU. Currently, John 1.7 achieves up to 1.6M c/s raw performance (that is, with no matching salts) on a PowerPC G5 at 2.7 GHz (or 1.1M c/s on a 1.8 GHz) and touches 1M c/s on the fastest AMD CPUs currently available. Intel P4s reach up to 800k c/s. (A non-public development version making use of SSE also reaches 1M c/s on an Intel P4 at 3.4 and 3.6 GHz. I intend to include that code into a post-1.7 version.)

John the Ripper makes password cracking easy Read More »

Google on the Google File System (& Linux)

From Sanjay Ghemawat, Howard Gobioff, & Shun-Tak Leung’s “The Google File System“:

We have designed and implemented the Google File Sys- tem, a scalable distributed file system for large distributed data-intensive applications. It provides fault tolerance while running on inexpensive commodity hardware, and it delivers high aggregate performance to a large number of clients. …

The file system has successfully met our storage needs. It is widely deployed within Google as the storage platform for the generation and processing of data used by our ser- vice as well as research and development efforts that require large data sets. The largest cluster to date provides hun- dreds of terabytes of storage across thousands of disks on over a thousand machines, and it is concurrently accessed by hundreds of clients. …

We have seen problems caused by application bugs, operating system bugs, human errors, and the failures of disks, memory, connectors, networking, and power sup- plies. Therefore, constant monitoring, error detection, fault tolerance, and automatic recovery must be integral to the system.

Second, files are huge by traditional standards. Multi-GB files are common. Each file typically contains many applica- tion objects such as web documents. When we are regularly working with fast growing data sets of many TBs comprising billions of objects, it is unwieldy to manage billions of ap- proximately KB-sized files even when the file system could support it. As a result, design assumptions and parameters such as I/O operation and blocksizes have to be revisited.

Third, most files are mutated by appending new data rather than overwriting existing data. Random writes within a file are practically non-existent. Once written, the files are only read, and often only sequentially. …

Multiple GFS clusters are currently deployed for different purposes. The largest ones have over 1000 storage nodes, over 300 TB of diskstorage, and are heavily accessed by hundreds of clients on distinct machines on a continuous basis. …

Despite occasional problems, the availability of Linux code has helped us time and again to explore and understand system behavior. When appropriate, we improve the kernel and share the changes with the open source community.

Google on the Google File System (& Linux) Read More »

The original description of Ajax

From Jesse James Garrett’s “Ajax: A New Approach to Web Applications“:

Ajax isn’t a technology. It’s really several technologies, each flourishing in its own right, coming together in powerful new ways. Ajax incorporates:

  • standards-based presentation using XHTML and CSS;
  • dynamic display and interaction using the Document Object Model;
  • data interchange and manipulation using XML and XSLT;
  • asynchronous data retrieval using XMLHttpRequest;
  • and JavaScript binding everything together.

The classic web application model works like this: Most user actions in the interface trigger an HTTP request back to a web server. The server does some processing — retrieving data, crunching numbers, talking to various legacy systems — and then returns an HTML page to the client. It’s a model adapted from the Web’s original use as a hypertext medium, but as fans of The Elements of User Experience know, what makes the Web good for hypertext doesn’t necessarily make it good for software applications. …

An Ajax application eliminates the start-stop-start-stop nature of interaction on the Web by introducing an intermediary — an Ajax engine — between the user and the server. It seems like adding a layer to the application would make it less responsive, but the opposite is true.

Instead of loading a webpage, at the start of the session, the browser loads an Ajax engine — written in JavaScript and usually tucked away in a hidden frame. This engine is responsible for both rendering the interface the user sees and communicating with the server on the user’s behalf. The Ajax engine allows the user’s interaction with the application to happen asynchronously — independent of communication with the server. So the user is never staring at a blank browser window and an hourglass icon, waiting around for the server to do something. …

Every user action that normally would generate an HTTP request takes the form of a JavaScript call to the Ajax engine instead. Any response to a user action that doesn’t require a trip back to the server — such as simple data validation, editing data in memory, and even some navigation — the engine handles on its own. If the engine needs something from the server in order to respond — if it’s submitting data for processing, loading additional interface code, or retrieving new data — the engine makes those requests asynchronously, usually using XML, without stalling a user’s interaction with the application.

The original description of Ajax Read More »

Embarassing email story #1056

From MedZilla’s “Emails ‘gone bad’“:

In another example of embarrassing and damaging emails sent during work is an investigation that uncovered 622 emails exchanged between Arapahoe County (Colo.) Clerk and Recorder Tracy K. Baker and his Assistant Chief Deputy Leesa Sale. Of those emails, 570 were sexually explicit. That’s not the only thing Baker’s lawyers are having to explain in court. Seems the emails also revealed Baker might have misused public funds, among other things.

Embarassing email story #1056 Read More »

Bruce Schneier on what we should do

From Bruce Schneier’s “Searching Bags in Subways“:

Final note: I often get comments along the lines of “Stop criticizing stuff; tell us what we should do.” My answer is always the same. Counterterrorism is most effective when it doesn’t make arbitrary assumptions about the terrorists’ plans. Stop searching bags on the subways, and spend the money on 1) intelligence and investigation — stopping the terrorists regardless of what their plans are, and 2) emergency response — lessening the impact of a terrorist attack, regardless of what the plans are. Countermeasures that defend against particular targets, or assume particular tactics, or cause the terrorists to make insignificant modifications in their plans, or that surveil the entire population looking for the few terrorists, are largely not worth it.

Bruce Schneier on what we should do Read More »

Religion & evolution

From Salon’s “Religious belief itself is an adaptation“, an interview with Edward O. Wilson:

Religious belief itself is an adaptation that has evolved because we’re hard-wired to form tribalistic religions. Religion is intensely tribalistic. A devout Christian or Muslim doesn’t say one religion is as good as another. It gives them faith in the particular group to which they belong and that set of beliefs and moral views. …

You cannot explain the patterns of diversity in the world, the geography of life, the endless details of distribution, similarity and dissimilarity in the world, by any means except evolution. That’s the one theory that ties it together. It is very hard to see how traditionalist religious views will come to explain the meaning of life on this planet. …

Religion & evolution Read More »

Phishing by altering the bank’s server

From Computerworld‘s “Florida banks hacked in new spoofing attack“:

Three Florida banks have had their Web sites compromised by hackers in an attack that security experts are calling the first of its type.

Earlier this month, attackers were able to hack servers run by the Internet service provider that hosted the three banks’ Web sites. They then redirected traffic from the legitimate Web sites to a bogus server, designed to resemble the banking sites, according to Bob Breeden, special agent supervisor with the Florida Department of Law Enforcement’s Computer Crime Center.

Users were then asked to enter credit card numbers, PINs and other types of sensitive information, he said.

According to Breeden, the affected banks are Premier Bank, Wakulla Bank and Capital City Bank, all small, regional banks based in Florida.

This attack was similar to phishing attacks that are commonly used against online commerce sites, but in this case hackers had actually made changes to legitimate Web sites, making the scam much harder for regular users to detect.

… Though Breeden said the scam was operational for only “a matter of hours” and probably affected fewer than 20 banking customers, the technique appeared to be very effective at extracting sensitive information.

Phishing by altering the bank’s server Read More »

Architecture & the quality without a name

From Brian Hayes’ “The Post-OOP Paradigm“:

Christopher Alexander [a bricks-and-steel architect] is known for the enigmatic thesis that well-designed buildings and towns must have “the quality without a name.” He explains: “The fact that this quality cannot be named does not mean that it is vague or imprecise. It is impossible to name because it is unerringly precise.”

Architecture & the quality without a name Read More »

A very brief history of programming

From Brian Hayes’ “The Post-OOP Paradigm“:

The architects of the earliest computer systems gave little thought to software. (The very word was still a decade in the future.) Building the machine itself was the serious intellectual challenge; converting mathematical formulas into program statements looked like a routine clerical task. The awful truth came out soon enough. Maurice V. Wilkes, who wrote what may have been the first working computer program, had his personal epiphany in 1949, when “the realization came over me with full force that a good part of the remainder of my life was going to be spent in finding errors in my own programs.” Half a century later, we’re still debugging.

The very first programs were written in pure binary notation: Both data and instructions had to be encoded in long, featureless strings of 1s and 0s. Moreover, it was up to the programmer to keep track of where everything was stored in the machine’s memory. Before you could call a subroutine, you had to calculate its address.

The technology that lifted these burdens from the programmer was assembly language, in which raw binary codes were replaced by symbols such as load, store, add, sub. The symbols were translated into binary by a program called an assembler, which also calculated addresses. This was the first of many instances in which the computer was recruited to help with its own programming.

Assembly language was a crucial early advance, but still the programmer had to keep in mind all the minutiae in the instruction set of a specific computer. Evaluating a short mathematical expression such as x 2+y 2 might require dozens of assembly-language instructions. Higher-level languages freed the programmer to think in terms of variables and equations rather than registers and addresses. In Fortran, for example, x 2+y 2 would be written simply as X**2+Y**2. Expressions of this kind are translated into binary form by a program called a compiler.

… By the 1960s large software projects were notorious for being late, overbudget and buggy; soon came the appalling news that the cost of software was overtaking that of hardware. Frederick P. Brooks, Jr., who managed the OS/360 software program at IBM, called large-system programming a “tar pit” and remarked, “Everyone seems to have been surprised by the stickiness of the problem.”

One response to this crisis was structured programming, a reform movement whose manifesto was Edsger W. Dijkstra’s brief letter to the editor titled “Go to statement considered harmful.” Structured programs were to be built out of subunits that have a single entrance point and a single exit (eschewing the goto command, which allows jumps into or out of the middle of a routine). Three such constructs were recommended: sequencing (do A, then B, then C), alternation (either do A or do B) and iteration (repeat A until some condition is satisfied). Corrado Böhm and Giuseppe Jacopini proved that these three idioms are sufficient to express essentially all programs.

Structured programming came packaged with a number of related principles and imperatives. Top-down design and stepwise refinement urged the programmer to set forth the broad outlines of a procedure first and only later fill in the details. Modularity called for self-contained units with simple interfaces between them. Encapsulation, or data hiding, required that the internal workings of a module be kept private, so that later changes to the module would not affect other areas of the program. All of these ideas have proved their worth and remain a part of software practice today. But they did not rescue programmers from the tar pit.

Object-oriented programming addresses these issues by packing both data and procedures—both nouns and verbs—into a single object. An object named triangle would have inside it some data structure representing a three-sided shape, but it would also include the procedures (called methods in this context) for acting on the data. To rotate a triangle, you send a message to the triangle object, telling it to rotate itself. Sending and receiving messages is the only way objects communicate with one another; outsiders are not allowed direct access to the data. Because only the object’s own methods know about the internal data structures, it’s easier to keep them in sync.

You define the class triangle just once; individual triangles are created as instances of the class. A mechanism called inheritance takes this idea a step further. You might define a more-general class polygon, which would have triangle as a subclass, along with other subclasses such as quadrilateral, pentagon and hexagon. Some methods would be common to all polygons; one example is the calculation of perimeter, which can be done by adding the lengths of the sides, no matter how many sides there are. If you define the method calculate-perimeter in the class polygon, all the subclasses inherit this code.

A very brief history of programming Read More »

Bertrand Russell on writing well

From Bertrand Russell’s “How I Write“:

Until I was twenty-one, I wished to write more or less in the style of John Stuart Mill. … I had, however, already a different ideal, derived, I suppose, from mathematics. I wished to say everything in the smallest number of words in which it could be said clearly. … I would spend hours trying to find the shortest way of saying something without ambiguity, and to this aim I was willing to sacrifice all attempts at aesthetic excellence.

At the age of twenty-one, however, I came under a new influence that of my future brother-in-law, Logan Pearsall Smith. He was at that time exclusively interested in style as opposed to matter. His gods were Flaubert and Walter Pater, and I was quite ready to believe that the way to learn how to write was to copy their technique. He gave me various simple rules, of which 1 remember only two: “Put a comma every four words”, and “never use ‘and’ except at the beginning of a sentence”. His most emphatic advice was that one must always re-write. I conscientiously tried this, but found that my first draft was almost always better than my second. This discovery has saved me an immense amount of time. I do not, of course, apply it to the substance, but only to the form. When I discover an error of an important kind I re-write the whole. What I do not find is that I can improve a sentence when I am satisfied with what it means.

… In fact, all imitation is dangerous. Nothing could be better in style than the Prayer Book and the Authorized Version of the Bible, but they express a way of thinking and feeling which is different from that of our time. A style is not good unless it is an intimate and almost involuntary expression of the personality of the writer, and then only if the writer’s personality is worth expressing. But although direct imitation is always to be deprecated, there is much to be gained by familiarity with good prose, especially in cultivating a sense for prose rhythm.

There are some simple maxims-not perhaps quite so simple as those which my brother-in-law Logan Pearsall Smith offered me-which I think might be commanded to writers of expository prose. First: never use a long word if a short word will do. Second: if you want to make a statement with a great many qualifications, put some of the qualifications in separate sentences. Third: do not let the beginning of your sentence lead the reader to an expectation which is contradicted by the end.

Bertrand Russell on writing well Read More »

Free markets need visibility to work

From Slashdot’s “Pay-per-email and the ‘Market Myth’“:

But I think there’s a bigger problem underlying all of this. It’s not about specific problems with GoodMail’s or AOL’s or Hotmail’s system. The problem is that many advocates of these systems say that any flaws will get sorted out automatically by “the market” — and in this case I think that is simply wrong. And in fact the people on Thursday’s panel can’t really believe it either, because one thing we all agreed on was that Bonded Sender sucks. But has the marketplace punished Hotmail for using it? Have people left in droves because non-Bonded-Sender e-mail gets blocked? No, because if they never see it getting blocked they don’t know what happens. Free markets only solve problems that are actually visible to the user.

Free markets need visibility to work Read More »

How much does stolen identity info cost?

From The New York Times‘ “Countless Dens of Uncatchable Thieves“:

In the online world, he operates under the pseudonym Zo0mer, according to American investigators, and he smugly hawks all manner of stolen consumer information alongside dozens of other peddlers at a Web site he helps manage.

“My prices are lowers then most of other vendors have and I will deliver them in real time,” reads a typically fractured Zo0mer post.

At the same forum, another user, “tabbot,” offers “any U.S. bank accounts” for sale.

“Balance from 3K and above: $40,” he writes. “Regular brokerage accounts from 3K and above: $70.”

Tabbot also offers full access to hacked accounts from credit unions. One, with a $31,000 balance, is being sold for $400. “I can try search specific info such as signature, ssn, dob, email access,” tabbot writes. “Account with an extra info will be more expensive.”

How much does stolen identity info cost? Read More »

Some thoughts on strong & weak social ties

From Ross Mayfield’s “The Weakening of Strong Ties“:

Mark Granovetter’s seminal paper, The Strength of Weak Ties (summary), revealed the difference between friends and acquaintances and how useful acquaintances can be for certain tasks like finding a job. The difference between a strong tie and weak tie can generally be revealed by time commitment underpinning the relationship. Strong ties are better for action, weak ties for new information.

But time has changed with new tools and social networking models at our disposal. For the first time many social networks are being made explicit, often without the knowledge of participants, at an accelerating pace and dramatically lowered search costs. This newfound transparency may very well make strong ties weaker. …

Top-down models, or weak tie systems, are particularly good at the breadth of sample and search. …

That said, weak tie systems provide real insight we didn’t have before. They have vast potential for revealing structural holes and other methods of enhancing social capital. So long as participation is voluntary, information flows monitored is public and constraints keep in check how people act upon intelligence these systems will become a critical competitive advantage for any company.

By contrast, bottom-up systems are built for people instead of companies. Within Friendster, Ryze, Tribe.net and LinkedIn connections are made through individual decisions upon of invitations, requests and confirmations. The network grows organically, decision by voluntary decision.

Some thoughts on strong & weak social ties Read More »

Why we don’t have rights from the ground to the sky

From Salon’s “Throwing Google at the book“:

Lawrence Lessig, a Stanford law professor and copyright scholar, likes to tell the story of Thomas Lee and Tinie Causby, two North Carolina farmers, who in 1945 cast themselves at the center of a case that would redefine how society thought of physical property rights. The immediate cause of the Causbys’ discomfort was the airplane; military aircraft would fly low over their land, terrifying their chickens, who flew to their death into the walls of the barn. As the Causbys saw it, the military aircraft were trespassing on their land. They claimed that American law held that property rights reached ‘an indefinite extent, upwards’; that is, they owned the land from the ground to the heavens. If the government wanted to fly planes over the Causbys’ land, it needed the Causby’s permission, they insisted.

The case, in time, came to the Supreme Court, where Justice William O. Douglas, writing for the Court, was not kind to the Causbys’ ancient interpretation of the law. Their doctrine, he said, “has no place in the modern world. The air is a public highway, as Congress has declared. Were that not true, every transcontinental flight would subject the operator to countless trespass suits. Common sense revolts at the idea. To recognize such private claims to the airspace would clog these highways, seriously interfere with their control and development in the public interest, and transfer into private ownership that to which only the public has a just claim.”

… the airplane rendered the Causbys’ rights to the skies incompatible with the modern world …

Why we don’t have rights from the ground to the sky Read More »

Identity production & sharing during adolescence

From danah boyd’s “Friendster lost steam. Is MySpace just a fad?“:

No, it is not just a moral panic that could make MySpace a fad. The primary value right now has to do with identity production and sharing, practices that are more critical to certain populations at certain times in their lives and it is possible that “growing up” will be marked by leaving MySpace (both for the teens and the 20-somethings).

Identity production & sharing during adolescence Read More »

Social capital: subcultural vs cultural

From danah boyd’s “Friendster lost steam. Is MySpace just a fad?“:

What’s at stake here is what is called “subcultural capital” by academics. It is the kind of capital that anyone can get, if you are cool enough to know that it exists and cool enough to participate. It is a counterpart to “cultural capital” which is more like hegemonic capital. That was probably a bit too obscure. Let me give an example. Opera attendance is a form of cultural capital – you are seen as having money and class and even if you think that elongated singing in foreign languages is boring, you attend because that’s what cultured people do. You need the expensive clothes, the language, the body postures, the social connects and the manners to belong. Limitations are economic and social. Rave attendance is the opposite. Anyone can get in, in theory… There are certainly hodgepodged clothes, street language and dance moves, but most folks can blend in with just a little effort. Yet, the major limitation is knowing that the rave exists. “Being in the know” is more powerful than money. You can’t buy your way into knowledge of a rave.

“Coolness” is about structural barriers, about the lack of universal accessibility or parsability. Structural hurdles mean people put in more effort to participate. It’s kinda like the adventure of tracking down the right parking lot to get the bus to go to the rave. The effort matters. Sure, it weeds some people out, but it makes those who participate feel all the more validated. Finding the easter egg, the cool little feature that no one knows about is exciting. Learning all of the nooks and crannies in a complex system is exhilarating. Figuring out how to hack things, having the “inside knowledge” is fabu.

Often, people don’t need simplicity – they want to feel proud of themselves for figuring something out; they want to feel the joy of exploration. This is the difference between tasks that people are required to do and social life. Social life isn’t about the easy way to do something – it’s about making meaning out of practice, about finding your own way.

Social capital: subcultural vs cultural Read More »

Intel: anyone can challenge anyone

From FORTUNE’s “Lessons in Leadership: The Education of Andy Grove“:

[Intel CEO Andy] Grove had never been one to rely on others’ interpretations of reality. … At Intel he fostered a culture in which “knowledge power” would trump “position power.” Anyone could challenge anyone else’s idea, so long as it was about the idea and not the person–and so long as you were ready for the demand “Prove it.” That required data. Without data, an idea was only a story–a representation of reality and thus subject to distortion.

Intel: anyone can challenge anyone Read More »