I recently talked to Joe Stump, CTO of SimpleGeo, about a number of topics related to location and databases. In the course of the interview, we also got around to discussing Digg. Previous to launching SimpleGeo, Stump was the lead architect at Digg, and he has a lot of insight into where the site is heading. We'll be running the rest of the interview soon, but what Stump told me about Digg got me thinking.We've all heard about citizen journalism. Digg, in principle, is citizen editing. One of the primary jobs of an editor is to decide what's important, and what's dross. Digg uses crowdsourcing to determine what rises above the noise to move onto the topic and main front pages. Being "dugg" can make or break a geek-related news story.But the general consensus these days is that Digg has been gamed into uselessness. Gangs of Digg assassins work to vote stories down that deal with topics they dislike. Mac folks slam Windows stories. Windows guys sink Linux stories. Cabals of content creators try to work together to vote each other's work up onto the front page. It's like a guerrilla war fighting to control the front page of your local newspaper.According to Stump, Digg is aware of this and plans to address it in an upcoming rework of the site. "Digg has always aggressively pursued ways of evening the field and avoiding those turf warfares," Stump said. "I think the next iteration is going to do a lot to answer that problem, because really you need to build consensus."What Stump believes is going to happen is that Digg is going to move to more of a recommendation model than a universal up/down voting system:I think the way that Digg is going to answer it, and the way that the internal thoughts were when I was there, is that rather than allowing a small group to yank a story from the bigger group, that we give people better tools so they don't even see those stories to begin with. If all you're going to do is bury Palin stories or Obama stories, maybe the answer isn't to figure out that you're a Palin or an Obama hater and to not count your buries, but maybe the better answer is to give you a tool where you can say, 'Screw that Obama guy. I don't want to see anything about him.' Or, 'Screw Palin. I don't want to see anything about her.' I think that at some point in the future, you'll probably see where those negative votes carry a much more personal connotation as opposed to a group connotation.Delivering more personalized news will probably be great for Digg users. But without a single front page to vie for, it will also blunt the power Digg has to make or break stories. "That's fed a lot of the problems that Digg's had up to this point," said Stump. "If there's not one unified front page, and everybody's front page is different based on their behavior and their interests and their niche categories, it removes a lot of the incentive to get something on the front page. Because you really don't know whose front page you're on once you get promoted."
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/dhjBtSfAdCQ/joe-stump-talks-about-the-futu.html
Oracle's acquisition of Sun has opened up all sorts of questions: Will MySQL get the support it needs? What will become of the MySQL community? Where should database administrators put their efforts and resources?Ronald Bradford can answer that last question. Bradford, an RDBMS expert and a speaker at the upcoming MySQL Conference and Expo, has been guiding DBAs through key aspects of MySQL integration for years. In the following Q&A, he discusses the pros and cons of migrating from Oracle to MySQL (hint: it's not just about cost savings). He also weighs in on the future of MySQL and its community.Jumping from Oracle to MySQLMac Slocum: What are the upsides to migrating from Oracle to MySQL? Is cost the major factor?Ronald Bradford:The Oracle license cost is generally the most important factor for organizations considering migrations. Also, integration with open source LAMP products that provide many features you see today, including project management, bug tracking, wikis, blogs, and customer relationship management, are better served when all systems can communicate with the underlying data storage in MYSQL.Newer and cheaper multi-core hardware, and a correct scale-out architecture, manages risk better then a single, large, scale-up instance of your data. Failure of 1-10 percent of your user data is far better then 100 percent failure.MS: What are the major issues with Oracle to MySQL migration?RB: Adequate education and skills development is the most significant and most under-budgeted cost in migration. While the cost of licensing and subscriptions is generally less for non-Oracle solutions, MySQL is not Oracle. Most organizations underestimate the time needed for staff to become proficient in a new skill, especially when they're required to maintain existing systems.The second factor is staff pushing back against MySQL. For example, management at a top 20 website I was involved with made the decision to replace Oracle with MySQL. The technical resources, including system architects and senior DBAs, were not in agreement and they sometimes actively fought against the implementation of MySQL.The third issue is monitoring. MySQL does not have the level of in-depth instrumentation it should. While it's integrated into existing open source monitoring products, MySQL is not always well supported by production network operations center systems.For these reasons, slow integration with less critical systems is a successful integration model. This enables time for a comfortable and successful transition and it creates confidence moving forward.
MySQL's futureMS: What do you think will happen to MySQL once dust settles from Oracle's acquisition of Sun?RB: Oracle has made a written commitment to invest in MySQL at the same rate as Sun for the next three years. Given the vast experience and R&D budget Oracle has given its product line, I hope more is invested in MySQL. In the year following MySQL's first acquisition by Sun, they failed to produce advances the community would have liked. Many in the community were disappointed by the stealth release of 5.4. I suspect it will take some time for Oracle resources, processes and procedures to integrate with the existing MySQL engineering lifecycle, and it will take time before we see any future work. I would like to see two areas of MySQL deficiency addressed: a totally integrated and online backup solution, and better instrumentation.MS: Will the MySQL developers under Oracle build stronger bonds with users? RB: Open-source culture is unique in comparison to commercial, closed-source products. Developers who work for MySQL now will continue to be available to alumni and the community, and I suspect they will continue to write, blog and actively present.While there has already been a small exodus of staff from the Oracle acquisition, any restrictions between staff and open source developers will greatly hurt the community. On the flip side, it will be difficult for Oracle developers who start using and contributing to MySQL development to be more open and responsive. It's unfamiliar ground.MS: Will work by Monty Widenius and others bring into being a larger after-market of MySQL extensions and patches?RB: The after-market movement started many years ago when MySQL executives lost perspective on the community and focused on commercial viability. This after-market will continue to flourish. I agree with a fellow community member who recently stated that the trademarked version of MySQL will be just one variant of the overall MySQL product. There are individual products that are as good if not better then the official MySQL version. What's lacking are commercial support options and documentation.MS: Will the varied projects that fall under NoSQL take market share from relational databases? Or will they grow in parallel?RB: MySQL has continued to grow in the database market. Oracle and other products have also grown. That indicates a general increase in demand regardless of the specific product. I see NoSQL adding value and options to an increasing marketplace.The NoSQL options have some great benefits, and in any emerging topology it's important to architect a solution that maximizes the strengths and minimizes the weaknesses of the products in use. Persistent or non persistent key-value stores, including Memcached, Tokyo Tyrant, Redis and Cassandra, are ideally suited for some functionality. Unstructured data being managed by MongoDB or CouchDB will be ideal in other situations.Another key area is that of free text searching. Products such as Sphinx, Lucene and Solr are critical to a successful website or application.The recent news that Twitter is moving to Cassandra highlights that products like these can deliver in highly available and scalable situations. It all depends on the right product for the given business requirements.MS: You'll be speaking at the MySQL Conference and Expo in April. What are you hoping attendees will take away from your "MySQLCamp for Oracle DBAs" tutorials?RB: It will be difficult in two sessions to provide great depth for Oracle DBAs. However, attendees will get the wealth of my 20 years of commercial experience in Oracle, MySQL and other RDBMS products. They'll also get a methodology for how to approach MySQL from a different skills background. Attendees will be able to identify the clear strengths and weaknesses of MySQL, and I'll provide information on common pitfalls. If I was to give a single statement that also encompasses some important content, it's this: Don't Assume. MySQL is not Oracle. Obvious mistakes in terminology, syntax and structure can be significant in the performance and best practices for using MySQL.The outline of the four talks in my MySQLCamp for the Oracle DBA series, two of which are being presented at the conference, is available on my website.Note: This interview was condensed and edited.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/-xYVHVqNC48/oracle-to-mysql.html
Foursquare is an on-the-rise application that blends mobile, location awareness and a clever points system that's an evolutionary leap for loyalty programs. Think coupons, but with rich data and gaming thrown in.Dennis Crowley, co-founder of Foursquare and a speaker at the upcoming Where 2.0 conference, cut his teeth on location services at Dodgeball, a mobile/social company that was a bit ahead of its time. Google acquired Dodgeball in 2005 but shuttered it in 2009. Crowley used that early experience -- the good and the bad -- to shape Foursquare.In the following interview, he digs into those Dodgeball lessons while also revealing Foursquare's revenue plans and the "secret sauce" that drives the service.Unlocking growth with local offersMac Slocum: Did Foursquare have a slow adoption pattern at the beginning and then hit a point where growth exploded?Dennis Crowley: It was super slow. We launched in March [2009] at South by Southwest and maybe picked up 2,500 users. It didn't take off the way we wanted it to when people got back from South by Southwest, and I think a lot of it was because the product was half-baked. Things improved in July when we got the iPhone version fixed a little bit and we started experimenting with the specials we're doing with local businesses. Things like, if you're the mayor, you get a free cup of coffee or a free slice of pizza. Once we started that, it helped tell our story better. December is when it really started to pick up. The growth curve is absurd right now. MS: Who is the typical Foursquare user?DC: We don't have that in terms of metrics because we don't collect age or income information. But I can make some generalizations about it, and I think people in the 24-35 age range represent the active user base. It's younger and social and probably a little bit more urban. But it definitely extends into the suburbs. We see parents with kids using it to connect with other parents at playgrounds. We see high school students using it. We see college kids using it.Allowing for adaptationMS: How has Foursquare adapted since launch?DC: Foursquare was launched almost as a response to Google turning off Dodgeball. Naveen [Selvadurai] and I wanted to build something to replace Dodgeball because there's nothing else that really works like that. So the initial functionality was built around check-in: you know where people are, and we'll be able to serve up recommendations. But then we started thinking about the things you'd want to build on top of that platform. How about little bits of information? Tips about the place that I'm at? Or, let's use game mechanics to push people to do things they normally wouldn't do and encourage people to have more interesting experiences. The game mechanics are the secret sauce. They keep people engaged long enough to see the interesting things that happen when they participate frequently. It's kind of like with Twitter. If you drop someone in Twitter and don't give them a reason to participate, they get bored of it really quickly. But, if you spend 10 days with Twitter, you fall in love with it. Foursquare is similar. Spend an afternoon with it, you'll say: "This is awful. I get nothing out of it." But as you start to get friends on it and as you check-in at different places, you realize complexities emerge. You see how people are using it and the content they've added. The game mechanics hold peoples' hands through the first 10 to 20 days of the service.
Foursquare's game mechanicsMS: Can you expand on the game mechanics? What aspects of Foursquare fall into that category?DC: It's anything that happens after the check-in. So many of these location services are just about the check-in. That's the end of the story. But the candy in Foursquare is every time you check-in, you get a couple points for doing something. You can become the mayor of a place if you've been there more than anyone else. Or in an even rarer situation, you unlock a badge because you've hit certain things in a certain order, or you've unlocked a certain pattern of usage: I've been to five food trucks. I've been to five karaoke places. I'm out really late on a Wednesday night. Weird things like that.I'm very inspired and motivated by Nike Plus. It's not a game, but there's game mechanics that make you run more. And I think Foursquare is going after the same thing. It's not a game, but the mechanics encourage you to experience things you normally wouldn't experience. I think we're just starting to scratch the surface of that space.MS: How have users changed Foursquare's functionality?We think of check-ins as you check-in at Yankee Stadium or you check-in at dinner. But then people use it to check-in to traffic, to check-in to the back of a cab. The playground is a good example. It's not a use case we anticipated. Parents uses it to meet up with other parents with kids. I think that's perfect. Use it whatever way you want.MS: Are user expectations growing beyond the boundaries of what you think is manageable?DC: I'm not so worried about the expectations because we have a really good idea of what we want to build. The thing is, we're still a relatively small team. We're dealing with scaling issues before we thought we'd have to. So, it's fine. We have to tackle those issues at some point, but we're really anxious to build because we have this great user base now. We want to push out all of these features, but first and foremost we have to make sure we keep the service up and running.
MS: Is interoperability a key to services like Foursquare? Do they have to work on as many networks as possible?DC: We thought in silos for a while, but now the tools to share content across networks are out there. So with Facebook allowing us to easily push content into Facebook -- and Twitter doing the same thing -- it just really helps spread the service. Fans create Foursquare mobile apps
MS: You relied on external developers to build a lot of your mobile apps. How did those relationships come about?DC: We built the iPhone one and then we rolled the dice a bit and built an API before anything else. That turned out to be a good bet because that's how we got the Android app. It started at South by Southwest where we met some kids who wanted to make an Android app for us. It took four months. There were three or four of them doing it in their spare time, which was great. And in a lot of ways, I think the Android app is better than the iPhone app that we built ourselves. But we built the API first and then we did a prototype of the Android app and then we tightened everything up from both sides. And now, because there's so much interest around our Android app, we brought an Android developer in-house to help manage the open source developers. MS: I'm really intrigued by businesses that have intentionally undefined boundaries. You nurture relationships with enthusiastic users who want to create something and then you make decisions off of that enthusiasm. DC: Yeah, and I guess in hindsight it seems like a really brilliant strategy. But it was all accidental. We barely got that iPhone app out for South by Southwest. I think we just got really lucky that we built something people wanted to use so badly that they wanted to build apps for it. MS: Who are your competitors?DC: People will say we're the next Twitter. Which I don't agree with. I think we're complementary to Twitter. Or, we're the next Facebook. But we're more of a feature set to Facebook. Or, we compete with Google Latitude. Well, kind of. But Foursquare's not just about the maps. It's about what happens nearby. And then people will say Yelp is a competitor because of the interest in local businesses and the ability to leave mini-reviews behind. I feel we're right in the middle of all of these folks.Foursquare's potential revenue streamsMS: What are you revenue plans? DC: Well, location-based advertising just doesn't feel right. The ads aren't relevant a lot of the time. I think that's going to work eventually, but there needs to be a better short-to-medium-term solution. Our answer has been scrappy promotions that local merchants can set up with us directly. And those are tied to the metrics of check-ins. So if you've been to a place five times, or if you're the mayor of this place, we take some of the data and turn it into a reward. Things like, because you're deemed a regular, you get a free slice of pizza or a free cup of coffee. This stuff isn't changing the world, but we're finding that venues like giving these rewards out and users love getting them.Now, is there an opportunity to monetize that little ecosystem we've created? Probably. I think it's going to take time to figure it out. We're not racing into that because we're encouraging venues to experiment with the basic tools we've created. We're seeing really interesting things come out of that. At the very beginning, it was come in and you get a free ticket to a band. Then it turned into a free cup of coffee. Then it turned into a free slice of pizza. And at this point, there's a hotel in Amsterdam that gives away a free hotel stay. There's a place in Texas that gives a free steak dinner. There's a place in New York that gives you a free bottle of wine. So we're getting to real, legitimate, interesting promotions. And it seems like it's maturing on its own. MS: But you guys aren't getting any percentage from those transactions, right? DC: We're not taking a cut of that yet. That's one of the things that comes up in conversation. People say we're going to kill it in the local coupon space. But I don't want to ruin it by nickel and diming local merchants.MS: That sounds like the Twitter approach. Don't hinder growth by pre-defining revenue streams.DC: It's a little bit. We look up to what Twitter has done. One thing that's different is we're doing experiments with brands and media companies earlier. What we don't want to do is go two years without any experiments in monetization and then, all of the sudden, have to flip a switch on. I think it's nice to have it built in from the very beginning, even if we're not generating revenue off of it.We're in a really fortunate position. We've had calls from large media companies, from newspapers, from TV stations, and TV shows and really, everything across the board. For example, Bravo TV is now on Foursquare. They wanted to do something where people go to places featured in their shows and unlock badges that are tied back to the shows. They're going to promote Foursquare on-air. So as you're watching it at home, it's like, "next time you're out in Atlanta or New York, check-in at these places to interact with Bravo in a different way." I thought that was an awesome idea. Different expectations for location dataMS: How does location data differ from status updates, pictures, links and other forms of shared information?DC: I guess it's pretty clear that we're all sharing more data than we thought of sharing before. But we've heard from the very beginning at Dodgeball that sharing location gets you in trouble sometimes. Things like, "I saw you were out with such and such when you actually said you were doing this." Or, your ex-girlfriend sends you a friend request and you have to approve it because you don't want to hurt her feelings, and then she knows where you are all the time. It creates awkward interactions. There's just something different about sharing location than sharing everything else because the whole point of location sharing is to enable real world meet ups. And if something goes wrong with location share, it enables the wrong types of real world meet ups. I think as more people start using these location services, they're going to be aware that maybe this is the one network that should just be about sharing with your real friends. The friends you would call to pick you up at the airport.Disclosure: O'Reilly AlphaTech Ventures is an investor in Foursquare.Note: This interview was condensed and edited.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/CyU4bZQO1oI/foursquare-location-apps.html
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/RYyRXydd7vg/google-buzz-is-it-project-prod.html
OSOR.eu -- The OSOR is a platform where public administrations can exchange information and experiences and collaborate in developing free and open source software. The platform has managed to bring together more than 2000 such open source software applications in just sixteen months after its launch. (via EUPractice and vikram_nz on Twitter)
Inside Glitch -- writeup of behind-the-scenes during the development of the game Glitch, the new project from Stewart Butterfield, Cal Henderson, Eric Costello, and Serguei Mourachov. The historical details themselves are banal, but what's interesting is how the reporter got access: "I'll let you determine when the piece runs (but not editorial control over what goes in it), and in return I get to meet regularly with you and you tell me all." It's analogous to the Newsweek tell-alls that come out after the election. (via Waxy)
Open Clip Art -- archive of public domain-contributed clip art. (via Mark Osbourne)
How To Split Up The US -- clique analysis from 210 million public Facebook profiles. Some of these clusters are intuitive, like the old south, but there's some surprises too, like Missouri, Louisiana and Arkansas having closer ties to Texas than Georgia. To make sense of the patterns I'm seeing, I've marked and labeled the clusters, and added some notes about the properties they have in common.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/kMf3c81ITkw/four-short-links-10-february-2.html
Kindle Development Kit APIs -- Amazon will release a Kindle SDK. These are the API docs. (via obra on Twitter)
rePublish -- all-Javascript ebook reader. (via kellan on Twitter)
Peer Review: What's it Good For? (Cameron Neylon) -- harsh and honest review of peer review with some important questions for the future of science. But there is perhaps an even more important procedural issue around peer review. Whatever value it might have we largely throw away. Few journals make referee’s reports available, virtually none track the changes made in response to referee’s comments enabling a reader to make their own judgement as to whether a paper was improved or made worse. Referees get no public credit for good work, and no public opprobrium for poor or even malicious work. And in most cases a paper rejected from one journal starts completely afresh when submitted to a new journal, the work of the previous referees simply thrown out of the window. Some lessons in here for social software, too.
Analog IMDB -- The transition is moving slowly, but it’s moving. It’s a fascinating thing to watch. The technology is the dull part: what’s interesting is the shift in perception. You know how sometimes you turn off a certain section of your brain and force yourself to see a word not as a piece of language with meaning, but as a sequence of black shapes and white spaces? It’s like you’re seeing that image for the very first time and suddenly “bird” seems like a very odd thing. I’ve been buying all of my in-print books electronically for a couple of years. Physical books aren’t weird to me yet. But damn, that old copy of the Maltin guide was a freaky and bizarre object. It’s the first time I looked at a book and didn’t see a container for information. I saw dead wood.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/Hmz8qej_Tys/four-short-links-8-february-20.html
I was talking recently with Bob Frankston, who has a distinguished
history in computing that goes back to work on Multics, VisiCalc,
and Lotus Notes. We were discussing some of the dreams of the Internet
visionaries, such as total decentralization (no mobile-system walls,
no DNS) and bandwidth too cheap to meter. While these seem impossibly
far off, I realized that computing and networking have come a long way
already, making things normal that not too far in the past would have
seemed utopian.
Flat-rate long distance calls
I remember waiting past my bedtime to make long-distance calls, and
getting down to business real quick to avoid high charges.
Conventional carriers were forced to flat-rate pricing by competition
from VoIP (which I'll return to later in the blog). International
calls are still overpriced, but with penny-per-minute cards available
in any convenience store, I don't imagine any consumers are paying
those high prices.
Mobile phone app stores
Not that long ago, the few phones that offered Internet access did so
as a novelty. Hardly anybody seriously considered downloading an
application to their phones--what are you asking for, spam and
fraudulent charges? So the iPhone and Android stores teaming with
third-party apps are a 180-degree turn for the mobile field. I
attribute the iPhone app store once again to competition: the uncovering
of the iPhone SDK by a free software community.
Downloadable TV segments
While the studios strike deals with Internet providers, send out
take-down notices by the ream, and calculate how to derive revenue
from television-on-demand, people are already getting the most popular
segments from Oprah Winfrey or Saturday Night Live whenever they want,
wherever they want.
Good-enough generic devices
People no longer look down on cheap, generic tools and devices. Both
in software and in hardware, people are realizing that in the long run
they can do more with simple, flexible, interchangeable parts than
with complex and closed offerings. There will probably always be a
market for exquisitely designed premium products--the success of Apple
proves that--but the leading edge goes to products that are just "good
enough," and the DIY movement especially ensures a growing market for
building blocks of that quality.
I won't even start to summarize Frankston's own writings, which start with
premises so far from what the Internet is like today that you won't be
able to make complete sense of any one article on its own. I'd
recommend the mind-blowing Sidewalks: Paying by
the Stroll if you want to venture into his world.But I'll mention one sign of Frankston's optimism: he reminded me that
in the early 1990s, technologists were agonizing over arcane
quality-of-service systems in the hope of permitting VoIP over
ordinary phone connections. Now we take VoIP for granted and are
heading toward ubiquitous video. Why? Two things happened in parallel:
the technologists figured out much more efficient encodings, and
normal demand led to faster transmission technologies even over
copper. We didn't need QoS and all the noxious control and overhead it
entails. More generally, it's impossible to determine where progress
will come from or how fast it can happen.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/Oc9GyuP1p3o/one-hundred-eighty-degrees-of.html
http://radar.oreilly.com/2010/01/the-chess-grandmaster-apples-i.html
Over the past three years I have been working to bridge gaps between the tech community & traditional emergency management organizations. I've focused on helping technologists adapt technologies to support humanitarian missions, often in response to a disaster. After Hurricane Katrina, Mikel Maron and I discovered a pattern for successful innovation during and after disasters. Understanding this pattern is crucial to "Serving Those that Serve Others".Pattern for DisasterTech Innovation 1. Disaster2. Ad-Hoc Adaptation 3. Championship 4. Iterative ImprovementThere is an unprecedented amount of interest and attention in finding ways to help in Haiti & around the world. The CrisisCamp & CrisisCommon projects are coordinating events and helping match organizations with needs to volunteers with skills. I encourage you to participate, and volunteer your time, knowledge, and resources. Serve those that serve others. You can make a difference now.Upcoming Crisis CampsSunday, January 24, 2010CrisisCamp Haiti, TorontoSaturday, January 30, 2010CrisisCamp Haiti, New YorkCrisisCamp Haiti, ChicagoCrisisCamp Haiti, MontrealSunday, January 31, 2010CrisisCamp Haiti, Washington DCCrisisCamp Haiti, New York (day 2)
http://radar.oreilly.com/2010/01/the-pattern-of-disaster-techno.html
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/CTWzh-WSPF8/haiti-tradui-translation-app-f.html
http://radar.oreilly.com/2010/01/haiti-tradui-translation-app-f.html
In the United States, Western Europe and Asia, e-Books are becoming a major player, especially now that e-Readers like the Kindle and Nook are available. But people living in the Arabic-speaking world or Africa haven't been invited to the dance. Two of the keynote speakers at the upcoming Tools of Change conference are working to improve access to e-Books in these areas: Arthur Attwell in South Africa and Ramy Habeeb in Egypt. We talked to each of them about how e-Books are important in their area of the world, and the challenges that they are facing.
Flash Player upgrade required
//
var wimpyConfigs38922a = new Object();
wimpyConfigs38922a.wimpySwf="http://cdn.oreilly.com/radar/rave/rave.swf";
wimpyConfigs38922a.wimpyApp="";
wimpyConfigs38922a.wimpyWidth="300";
wimpyConfigs38922a.wimpyHeight="80";
wimpyConfigs38922a.wimpyReg="NWtBMGhMbGdNJTdGJTNGb1IlMjV3JTNELUElM0FuWlolM0VmRkNIYXBa";
wimpyConfigs38922a.wimpySkin="http://cdn.oreilly.com/radar/rave/oreilly_skin_notar.xml";
wimpyConfigs38922a.autoAdvance="no";
wimpyConfigs38922a.playlist="http://cdn.oreilly.com/radar/2010/01/ArthurAttwell.mp3";
makeWimpyPlayer(wimpyConfigs38922a, "flashcontent38922a");
// ]]>
You may also download this file. Running time: 16:20
Arthur Attwell runs Electric Book Works, based out of Cape Town. His company does both traditional print publication and electronic publication, but he believes that e-Books have a particular promise in South Africa. "Certainly in South Africa, our traditional model doesn't even begin to reach the market that I think digital publishing could cater for. For me, digital is a massive social development tool. I like to think of e-books as one small application of digital publishing, which is really a grand process of putting the world of letters onto the internet. "Mobile is one of the keys to that, I think, for Africa because of the existing penetration of mobile devices, but there may be other ways of harnessing digital as well that will include distributing e-books through libraries and internet cafes, kiosks, any infrastructure that doesn't require someone to be spending a lot of money on a device. I think print on-demand has got a massive future for Africa, and developing countries in general, because of the way it caters to people with low cash flow and who just need a book right now; they can't afford to get an e-reader or even a netbook computer to read books in the long-term." "I think that we will see an incredible growth of digital publishing in Africa over the next few years, we're in the process right now of really just laying down the infrastructure that's going to make that possible. Mobile has done a lot, but because mobile tends to be controlled by network operators, it doesn't have quite the freedom of the internet. So I don't think it's necessarily going to see the same innovation at a very high centralized level. But I do think that with the massive growth of bandwidth and connectivity we're seeing right now, especially in Central Africa, that more conventional web-based applications of content and content-sharing will take off there as well."While mobile access to e-Books in Africa is largely an urban phenomenon right now, Attwell thinks that is changing. "You're probably going to find that 80 percent of internet connections in any African country will always be in the urban centers. So that's naturally then where the investment money's going to be going. But we're already seeing some exciting innovative approaches to getting internet connectivity into more rural areas. I know that in South Africa, we have fairly common solution where farmers in a particular area will get together and pool their resources to share a satellite internet connection or something similar, often even solar-powered connections. Naturally, rural is an area where mobile will be critical.""I think one of the really exciting trend-setting technologies at the moment is the success of the M-Pesa mobile payment system in Kenya. I think that that system is showing the power of a simple effective mobile application, there obviously for the purpose of transferring money between people. But it's an incredibly powerful tool in Kenya and used as much in rural areas as it is in urban."
One of the challenges Attwell faces is the issue of obtaining rights to translate works, something that currently requires laborious individual negotiations for each book. "What I'm going to be speaking about at Tools of Change this year is about encouraging publishers to license their work more flexibly. I think a translation license is one particular area where this is going to be important and powerful. I don't think that, for real market penetration in Africa, it's going to be possible to use the existing person-to-person, deal-by-deal negotiated translation agreements that the publishing industry is used to. It's not going to be a case of sitting across the table from someone at the London Book Fair signing a rights deal. You're going to need to have a very flexible way to allow local operators to translate your work into their languages for a very straightforward, standardized, quick, easy license transaction because ultimately, the language is one of the issues of last-mile delivery."
Ramy Habeeb faces very different issues with his project. Kotobarabia, which means Arabic Books, is trying to become the primary source for Arabic literature in electronic form. Habeeb started Kotobarabia because he found there was a real lack of e-Books available in the arabic language. He says that he never wants to see another situation like the Library at Alexandria, where one fire wiped out huge chunks of cultural wealth.But unreadable books could be as devastating as burned ones, so one of the challenges Kotobaradia faces is keeping all the content fresh. "As long as you're aware of the changes in the markets, you'll be able to keep up with it. An example of this is when we first launched Kotobarabia, we only did PDFs because PDFs were pretty much the only software display that was reasonably compatible with Arabic. Now, we created our own simple form of DRM to display our books, which you can see on our website. And very soon, we will be launching an e-pub version, the first Arabic e-pub book that I know of at least. So we are keeping up-to-date with the formats and we are trying to keep the content relevant." Another problem Habeeb faces is the non-uniformity of arabic texts. "One of the problems with Arabic e-books is that there is no OCR. Google claims that they have cracked the OCR nut, and if anyone can do it, it's Google. But I haven't yet actually seen that with my own eyes, to see how it works. Part of the reason why we have issues with OCR is because there are thousands of fonts that are usually customized to local publishing houses. It's almost like a signature of that publishing house to create their own font, it's part of the culture in publishing. Also, there are so many dots and lines and other things that an automated OCR system can mistake for a letter or distort into another letter. And to complicate matters even more, because the industry is relatively poor, the quality of paper and the quality of ink used isn't always the highest. All of these factors combined make OCR an extremely difficult endeavor. So as a result, whenever we take on a book, it either goes through one of two processes." "One process is that we fully type it so that it's fully searchable. We discovered that typing a book with a series of edits is actually cheaper than working with current OCR software that's on the market. Then we'll go through a whole process of creating the metadata behind it and uploading it to the site and converting it to the two formats that we are currently using commercially.""The thing that we do is to scan the pages, and then we'll have people read the pages and pick out key words so that the books become semi-searchable. We do these for most of our books. But if we find that a book is being read over and over again or that this title has a particular interest, then we'll go back and retype it. It's actually cheaper this way to do it, it's a more sustainable business model."Another issue Habeeb faces is that rights clearance can be very complicated. "We've had several cases where we've signed with publishing houses only to discover that the publishers never owned the e-rights, nor did the publishers really understand what e-rights were. So we tend to sign directly with authors, which is a real pain for us because we've signed with over 1,300 authors. That's a lot of contracts to keep in mind and follow-up with. It would be easier to sign with 200 publishing houses than 1,300 individuals. But it was the only way to ensure that we were honoring copyright law."Obviously, the issue of censorship is a huge one for Habeeb, but it's one he doesn't shy away from. "You have a choice when looking at a project like ours. And the choice is you either do it and bear the risk or you don't do it and you're happy with the status quo. So we, of course, have taken legal measures to best protect ourselves. But our rule of thumb is that just because the book has been censored doesn't mean that it's not valid, so we do have censored books on our site. But we also do take steps. For example, with Egyptian law, as long as the content is hosted from a server outside of Egypt, Egypt has no control over that server to ask you to shut it down. So that's why we have a US server. "Habeeb is aware that certain books are so controversial that they can cause problems. "Like the censorship game, it is a diplomatic game that needs to be played. We won't necessarily publish the book the day after it was published. We might wait a year for the attention on it to die down a little bit. It is a game that needs to be played, and we play it to protect ourselves. But ultimately, we believe in transparency and we believe in the free dissemination of information. We believe that information should be equally accessible to those who are interested in it." "Kotobarabia is myself and two other business partners, but the three of us aren't political crusaders who have some agenda in mind. We're just three people who have an appreciation for books, who love books and who want to share these books with the world. I'm always hesitant to come across as someone who has any political agenda."Censorship is more than a policy issue that Kotoarabia has to work with, it is also an ingrained mindset that can be insidious. there is a lot of self-censorship. "I have personal experience with that as well as knowledge of the market, that there is a political atmosphere of fear. If I write this down and it gets published and someone reads it, I could go to jail or I could get in trouble or I could bring problems to my family. And then there is also an issue of just lack of understanding. For example, the concept of free thought in the sense of that all ideas are okay is not very prevalent there.""For example, as I mentioned earlier, we type books. We had a case where there was one book written by a Muslim author about Egypt. And if I can remember correctly, he basically was writing a book about 1968 and saying that this year was paramount to modern Egypt, in how modern Egypt exists today. He would list events from this year from the newspapers, and one of the events that he listed was that in this year, an effigy of the Virgin Mary appeared on a few of the Catholic Church walls in Zaytoun. So we have these typists who were typing the books from A to Z, and then we have editors who will go in afterwards. It's common for a typist to miss a line or miss two lines, they're going so fast that their eyes just skip it. But this guy actually missed three pages, and when we looked closely at it, it was the three pages talking about the Virgin Mary effigy. And so when we questioned this guy about why these three pages were missing, he very innocently looked up at us and said, 'Oh, because it's not true so why write it?'" Both men are looking forward to attending Tools of Change in February. For Ramy, it's all about networking. "TOC is an incredible event for sharing of technologies and for seeing what the modern trends of publishing are. One of the things that we're looking at is, because the state of distribution in the Arab world is so dismal, we need to look at other forms of distribution, and e-publishing is the way. It's just a fantastic forum to sit down with people and discuss issues. I participated in the TOC event in February in Frankfurt, and I met Neelan Choksi, the head of Stanza. That was a fantastic eye-opening meeting where we sat and discussed the possibility of having an Arabic catalog in Stanza. Another person that I was introduced to by Andrew Savikas was Liza Daly, who was really fantastic in helping us figure out some of the issues with Unicode, and also understanding that in the Arab market, we just don't necessarily have the expertise to deal with some of these problems so we need to teach ourselves. Just being in an environment where you have people who understand publishing but also understand innovation, it's absolutely inspirational."Attwell agrees that TOC is a great resource for trading information. "I think that it's always helpful to get a whole lot of people in one room who've been thinking about the same kinds of issues. Perhaps it's simply because I'm kind of far away out here in Cape Town, but I often feel that those of us who think very deeply and hard about digital publishing issues are very often working in little silos either within our companies or as freelance consultants. Much of that thinking can be shared at a place like TOC, it's incredibly valuable to each of us personally and also to the publishers that attend. For me, one of the major things I get out of TOC is putting faces to names. I think that in the e-book digital publishing communities, everyone knows each other by their Twitter handles, and it makes a huge difference to actually sit down around tables with people and have real conversations. I think it can have a massively positive effect on our businesses, not just our enjoyment of our jobs."
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/8fLWiLgIqgI/bringing-e-books-to-africa-and.html
Telling Time with Open Realtime Data -- Sony Ericsson MBW-150 bluetooth watch, showing the next few SF Muni bus arrival times for a nearby stop. The code to fetch the arrival times is running on my Droid phone, and communicating with the watch using Marcel Dopita’s OpenWatch software for the Android platform. This is a neat hack, and reminds us that every object on our person could be programmed. (via Brian Jepson)
EZ430 Chronos -- wireless, programmable, pressure sensor, accelerometer, temperature sensor, all in a watch. (via Makezine)
Developing Bioinformatics Methods -- the best method developers, in general, are those people who are both developers and users of their own methods. Regardless of what field you're in, look for the alpha geeks: those who have both a problem and the means to solve it.
How to Innovate Using Existing Technology (Caterina Fake) -- interesting observation, that there's a sweet spot between "just a feature" and "needs ten years of basic research in academia" to get something that's defensible, useful, and achievable with the means of a startup. I'm a big fan of augmented human skill: using computers to make humans more effective at doing what humans are good at.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/ByGWyeopCFk/four-short-links-13-january-20.html
Introduction to Computational Advertising -- slides to a Stanford class on a new "scientific discipline" whose central challenge is to find the best ad to present to a user engaged in a given context, such as querying a search engine ("sponsored search"), reading a web page ("content match"), watching a movie, and IM-ing. "Scientific discipline" makes me gag. You could devise algorithms, measure performance, and write papers about the best way to put carrots up your bottom or the best way to pick pockets, but those still aren't complex enough activities to be trumpeted as "new scientific disciplines". (Although I do look forward to reading Stanford's CBUM126, "Introduction to Carrot Stuffing" lecture notes online). (via Greg Linden)
Timing Attack in Google KeyCzar Library -- if you compare strings in the naive way, attackers can figure out whether the first bytes they gave you are correct based on the time the comparison takes. When they get the first bytes correct, then they can work on the next, and so on. This is a common mode of information leakage, and reminds me of my revelation when I began to edit security books: "this stuff is hard". New programmers are not taught to think like attackers, and the only trope of secure programming that they're taught is "avoid buffer overflows". (via Simon Willison)
Climate Wizard -- explore historical temperature data as well as the various climate models and see what their predictions look like across the United States. (via Sciblogs)
Contextual Clothing for Naked Transparency (Jon Udell) -- notable for this: The Net can be an engine for context assembly, a wonderful phrase I picked up years ago from Jack Ozzie. We used to think that the challenge of social software was to amass as many users as quickly as possible, but the far harder problem to solve is how to help those people contribute to something positive. YouTube comments shows that simply having a lot of users doesn't make something virtuous.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/KX4YeXwjSRk/four-short-links-5-january-201.html
Recently, I wrote a post about Government 2.0 predictions for 2010-12, and one of them was that government would "always be on-the-record."By that I meant that the combination of (1) the proliferation of tech-savvy citizens with mobile camera/video devices, (2) the prevalence of wi-fi or other Web connections, (3) the massive number of people using social networks like MySpace, Facebook, and Twitter, and (4) the great interest that people have right now in a number of controversial issues like our current wars, health care, and climate change that people could and probably would start documenting everything that government officials do and say, where they go, who they meet with, for how long, what their staffers eat for lunch and with whom, and so on.And you don't need to be a professional journalist to do this, or even to do it well. An entire site along the lines of Gawker.com could be started around this, in fact. GovernmentGawker.com, anyone?Well, I was doing some research to look at planes versus trains to get home for the holidays (in light of the recent blizzard that's affected transport in the DC-NY-Boston corridor), and I came across a fantastic video that essentially puts the Amtrak Acela First Class service on the record for the trip between New York and Boston (7 min edited clip). Check it out.Now, imagine if someone did the same thing, but wanted to document a day in the life of Senator Ben Nelson, currently in the middle of heated debate about health care legislation. It's not hard. You check the general schedules of his committees and such beforehand, research powerful, under-the-radar staff and other relevant people on the Washington Post's WhoRunsGov.com, go through simple security at the Capitol (far easier than an airport), find Nelson's office in the Hart building, camp out in his waiting area, maybe ask the person at the front desk some questions, find some press in the hallways and ask some questions (maybe visit the Russell rotunda, where the television crews do their spots), stalk the cafeteria (there's a great coffee shop called Cups in the basement) and listen for people saying "Nelson," go back to his office and see him leaving to walk down the hall to a committee hearing, take photos of the staff with him on your Samsung ST-1000 with wi-fi and geo-tagging and upload the pics to Bing Maps and Facebook, go to the sub-committee hearing and tape it from a Flip in your coat pocket while you tweet live notes, upload your Flip video to YouTube while you follow Nelson to his next meeting, and so forth. (Note: This post has nothing to do with Sen. Nelson or health care, it's just an example "ripped from the headlines" - I've even met and chatted with him when he spoke about energy at the Defense Department, he's a nice person.)You can surely imagine at this point many variations on this for political appointees you don't like, lobbyists you're interested in, principal deputy assistant secretaries that make important decisions but don't necessarily travel in armored vehicles with bodyguards, various members of the press who might be meeting with sources at Capitol Hill bars, etc. Trust me, this isn't hard. If you live in Washington, DC, you probably realize how very easy this is, in fact, when combined with some good traditional news sources like the Post, Times, The Hill, and Politico. (If you live in Washington, DC, you also know that it's incredibly common to know where various officials live, eat, and so forth - I used to live about two blocks from Senator Obama's pad.) But why would someone want to create an "ambient stream" of Senator Nelson or anybody else's life? (Besides it being fascinating in a lowbrow Gossip Girl kind of way, of course.) Well, most people wouldn't. But so what? It's just like Wikipedia - only about 1% of people who use Wikipedia actively edit it; about 9% do sometimes, and 90% just read it. Twitter is not unlike that either - only about 10% of users contribute 90% of the tweets.So what if 1% of U.S. citizens started doing this? Roughly there are 300 million people in the U.S., say half of them are adults, so we have 1% of 150 million as 1.5 million. Now, if everyone just did this at the state, local, or federal level one day a year, and generated one "amateur journalism piece" from that day, that's about 4,100 videos/blog posts/tweet sets generated PER DAY. That's a lot of government on-the-record.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/c6Zpsd8f_xc/what-would-always-on-the-recor.html
In the circles that I travel the Internet is often breathlessly embraced as the herald of all things good; the bringer of increased choice, personal empowerment, social harmony...and the list goes on. And yet, as with any powerful technology, the truth of its consequences eludes such a singular and happy narrative.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/beaRhm1nIyQ/three-paradoxes-of-the-internet-age.html
ChipHacker -- collaborative FAQ site for electronics hacking. Based on the same StackExchange software as RedMonk's FOSS FAQ for open source software.
Democracy Live -- BBC launch searchable coverage of parliamentary discussion, using speech-to-text. One aspect we're particularly proud of is that we've managed to deliver good results for speech-to-text in Welsh, which, we're told, is unique. I think of this as the start of a They Work For You for video coverage. I'd love to be able to scale this to local government coverage, which is disappearing as local newspapers turn into delivery mechanisms for real estate advertisements.
InfiniDB: Open Source Column Database -- hooks into MySQL, uses MySQL for SQL parsing, security, etc. The commercial enterprise version has multi-server support (parallel scale-out). (via Brian Aker)
Massive Online Analysis -- MOA is a framework for data stream mining. Includes tools for evaluation and a collection of machine learning algorithms. Related to the WEKA project, also written in Java, while scaling to more demanding problems. . (via joshua on Delicious)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/0IQP5FMA_uk/four-short-links-4-november-20.html
Your Botnet is My Botnet (PDF) -- 2008 USENIX Security paper analysing >70G of data gathered when security researchers hijacked the Torpig botnet. A major limitation of analyzing a botnet from the inside is the limited view. Most current botnets use stripped-down IRC or HTTP servers as their command and control channels, and it is not possible to make reliable statements about other bots. In particular, it is difficult to determine the size of the botnet or the amount and nature of the sensitive data that is stolen. One way to overcome this limitation is to “hijack” the entire botnet, typically by seizing control of the C&C channel. [...] As a result, whenever a bot resolves a domain (or URL) to connect to its C&C server, the connection is redirected or sinkholed. This provides the defender with a complete view of all IPs that attempt to connect to the C&C server as well as interesting information that the bots might send..
cartographer.js -- build thematic maps using Google Maps. To be precise, you can build a choropleth, which is my word of the day. (via Simon Willison)
Making Privacy Policies Not Suck (Aza Raskin) -- interested in developing a standard set of privacy policy components the way that Creative Commons has created a standard set of copyright license components.
Scamville: The Social Gaming Ecosystem of Hell (TechCrunch) -- many of those games on Facebook that your friends play are evil. To get in-game money or objects, they'll let you take a survey but at the end you're signed up for crap you never wanted. Related: this article on monetizing social networks which talks about social gaming's business model.
http://radar.oreilly.com/2009/11/four-short-links-2-november-20.html
Recurring outages on major networking sites such as Twitter and
LinkedIn, along with incidents where Twitter members were
mysteriously dropped for days at a time,
have led many people to challenge the centralized control exerted by
companies running social networks. Whether you're a street
demonstrator or a business analyst, you may well have come to depend
on Twitter. We may have been willing to build our virtual houses on
shaky foundations might when they were temporary beach huts; but now
we need to examine the ground on which many are proposing to build our
virtual shopping malls and even our virtual federal offices.Instead of the constant churning among the commercial sites du
jour (Friendster, MySpace, Facebook, Twitter), the next
generation of social networking increasingly appears to require a
decentralized, peer-to-peer infrastructure. This article looks at
available efforts in that space and suggests some principles to guide
its development.Update: a few days ago, OpenID expert Chris Messina and
microblog developer Jyri Engeström published
an article with conclusions similar to mine;
clearly this is a felt need that's spreading across the Net.
Interestingly, they approach the questions from a list of what
information needs to be shared and how it needs to be transmitted; I
come from the angle of what people want from each other and how their
needs can be met. The two approaches converge, though. See the
comments for other interesting related blogs.
The peer-to-peer conceptThe Internet was originally a parliament convened among peers. Every
host was a server, almost always providing file downloads and usually
email as well. To this day, ISPs "peer" when they accept data from one
ISP's customer and delivers it to the other ISP's customer.To peer doesn't mean simply to be of equal status--in fact, that
notion could be misleading, because two systems with vastly different
roles and resources can peer. More importantly, to peer means to have
no intermediary.When the architecture requires an intermediary, it should play as
unobtrusive and minimal role as possible. For instance, Napster and
Skype have central servers, but they are used just to sign up
participants and set up connections among them.Napster's and Skype's partial decentralization won them a key benefit
of peer-to-peer networking that Twitter could well take note of: they
offload most traffic from their central servers to the users and the
ISPs that connect them.But being partially centralized means the service can still be
disrupted as a whole. Napster was shut down by a court ruling; Skype
shut itself down once through a programming error that it never
clearly explained to the public.The Internet itself quickly developed into this hybrid model as well.
Modems and terminals created a new layer of second-class citizens,
vastly expanded by the PC revolution. These Internet users were tucked
away behind firewalls and blocked from using any services not approved
by system administrators.By the year 2000, new companies springing up in the dot-com boom found
themselves frustrated by these restrictions, and designed their
innovative protocols to deliver data over port 80 because everybody
kept that open for Web traffic. When the practice started, traditional
Internet developers derided it as "port 80 pollution." Now it's called
Web Services.As happens so often, the way forward proved to be the way
backward--that is, to restore the democracy of the early Internet--and
also predictably, was pioneered by outlier movements with dubious
legality, ethics, and financial viability. Napster made the first
impact on public consciousness, followed by services that rigorously
avoided any hint of central servers (see my 2000 article,
Gnutella and Freenet Represent True Technological Innovation).By the end of 2000, the term peer-to-peer had become a
household word. But the movement quickly went into retreat, facing
difficult design problems that were already under discussion in the
O'Reilly book
Peer to Peer,
published in February 2001. I summarized the problems, which remain
ongoing, in the articles
From P2P to Web Services: Addressing and Coordination and
From P2P to Web Services: Trust.The issue of addressing would arise right away for a social network
developed in a pure peer-to-peer fashion. How would you check whether
your old college buddy was on the network, if you couldn't query a
central server? And how could you choose a unique name, without a
single place to register? Names would have to be qualified by domain
names or some other identifiers--which is actually a step forward
right there. It seems to me ridiculous that a company would plan to
provide a service to the whole world using a flat namespace. And while
we're at it, you ought to be able to change your name and bring along
all your prior activity.Trust would also become an issue in decentralized social networks. You
could ban a correspondent from your personal list, but you couldn't
inform a central authority about abuse. And the problem Twitter has
recently started to tackle--preventing random users from impersonating
well-known people--would be a challenge.But decentralization brings many benefits. A failure at one person's
site, or even on a whole segment of the network, would have no effect
on the rest of the world. A misconfigured router in Pakistan could not
keep everyone from accessing the most popular video content on the
Internet. And because each peer would have to obey common, understood
protocols, a decentralized social network would be transparent and
support the use of free software; nobody would have to puzzle over
what algorithms were in use.Visiting many different sites instead of central server to pull
together information on friends would increase network traffic, but
modern networks have enough bandwidth to stand up to the load. Even in
places with limited bandwidth, service would degrade gracefully
because messages would be small.The
StatusNet
project, which underlies
identi.ca,
represents a half-way step toward the kind of full decentralization
illustrated by RSS. StatusNet can power a variety of microbloggin
services, each signing up any number of members. The services can
interchange data to tie the members together.The rest of this article looks at two possible models for a
distributed social network (RSS and XMPP), followed by an examination
of the recurring problems of peer-to-peer in the social networking
context.Possible modelsMany examples can be found of filesystems, version control systems,
and other projects that lack central servers. But I'm just going to look
at two protocols that other people are considering for decentralized
social networking.When thinking of decentralized systems for sending short messages, RSS
and Atom have to come to mind first. They're universal and work well
on a large scale. And Dave Winer, the inventor of RSS, has created an
enhanced version called
rssCloud,
recently
incorporated into WordPress.Given the first question I asked about decentralization--how do you
find the people you're looking for?--the RSS answer is "by
serendipity." Like everything else on the Internet, you could come
across new treasures in many ways: surfing, searching, friends, or
media outlets. Lots of bloggers provide links from their sites to
their own faves. And RSS has developed its own ecosystem, sprouting
plenty of aggregators that offer you views into new fields of
information.rssCloud is meant to carry more frequent traffic and more content than
the original RSS and Atom. It maintains an XML format (making it
relatively verbose for SMS, although Winer tries to separate out the
rich, enhanced data). Perhaps because of the increased traffic it
would cause, it's less decentralized than RSS, storing updates in
Amazon S2.XMPP was invented about the same time as RSS by a programmer named
Jeremie Miller, who wanted a standard instant messaging protocol with
tags that could support semantics, and therefore powerful new
applications. Most important, his creation, Jabber, made it possible
for individual users to run their own servers instead of depending on
America Online or Yahoo!. Jabber had the potential to complement Tim
Berners-Lee's idea of a Semantic Web.Because Jabber used XML, it was seen as a bit heavyweight, and the
servers were reportedly hard to configure. But the possibilities were
too promising to pass up. So the IETF formalized it, gave it a clumsy
name suitable for a standard, and released a set of RFCs about it.
Unfortunately, XMPP languished until Google adopted it for their Talk
and Wave services. These high-profile applications suggest that it has
the scalability, flexibility, and robustness for social networking.The P2P problems, in today's contextEven if decentralized protocols and clients were invented, there will
be a long road to democratizing social networks. The messages are
expected to be lightweight, so photos and other large batches of
content would have to be stored somewhere outside the messages. Most
users wouldn't trust their laptops (much less their mobile devices) to
store content and serve it up 24 hours a day, so they would need a
cloud service, which might or might not be distributed.A backup service is also necessary in order to recover from a local
disk failure or other error that wipes out several years of your
accumulated identity.Problems such as impersonation and unsolicited communications (spam)
are hard to solve in decentralized systems because trust is always a
hierarchical quality. This is true everywhere in life, beyond the
level of a family or neighborhood. We expect our professors to be good
because they were hired by the college, and expect the college to be
good because it was accredited by a centralized facility, whose
managers were in turn appointed by elected officials. This system can
and does break down regularly, so mechanisms for repair are always
built in.Nobody can be banned from a decentralized social network because
there's nothing to ban them from. But there are ways to re-introduce
enough centralization to validate credentials. For instance, the
American Bar Association could register lawyers in good standing, and
you could check whether someone claiming to be a lawyer in the US was
registered. But we wouldn't want to take this process too far and
create a web of accreditations, because that would devalue people
whose skills and viewpoints lie outside the mainstream.You could still check whether someone shares friends with you, because
one person's claims of friendship could be traced back to the sites he
claims to be friends with. Someone could game the system by setting up
fake sites claiming to be people you know and linking back to them,
but this is a huge amount of work and leaves the perpetrator open to
arrest for fraud. Free software developer Thomas Lord suggests that
identity could also be verified through "a fairly shallow and
decentralized hierarchy of authentication like the system of notary
publics in physical life."All in all, the problems of finding people and trusting people
suggests that there's role for aggregators, just as in the case of
RSS. And these aggregators could also offer the kind of tracking
services (who talked about me today?) and statistical services (is
Michael Jackson's death still a major topic of conversation?) that get
business people so excited about Twitter. A decentralized social
network could still be business-friendly, because traffic could be
analyzed in order to target ads more accurately--but hopefully,
because peering clients are under the users' control, people who
didn't want the ads could configure their systems to screen them out.When you set up an account, you could register with aggregators of
your choice. And whenever you connected to someone, you could
automatically register his account with a list of your favorite
aggregators, in case he hadn't registered himself. If people wanted
control over where they're aggregated, I supposed something equivalent
to a robots.txt file could be invented. But it's not sporting
to refuse to be counted. And there's no point in invoking privacy
concerns--face it, if the NSA wants to read your tweets, they'll find
a way.So those are some of the areas where the problems of P2P and social
networking intersect. Let's remember that current social networks are
far from solving problems of findability, trust, and persistence as
well. I don't check how many followers I have on Twitter; I figure
most of them are spam bots. (Apologies to any of my followers who
actually are sufficiently embodied to be reading this.)Could
OpenSocial
be used to implement a P2P social network? It's based on a single
object that is expected to query and update a single server. But the
interface could probably be implemented to run on a single user's
system, registering the users or aggregators with whom she
communicates and querying all those users and aggregators as
necessary.Industry analysts have been questioning for years whether Twitter is
financially viable. Well, maybe it isn't--maybe this particular kind
of Internet platform is not destined to be a business. Responsibility
for the platform can be distributed among millions of sites and
developers, while business opportunities can be built on top of the
platform as services in analytics, publicity, and so forth.Like Google, Twitter and the other leading commercial Internet sites
have made tremendous contributions to the functionality of the
Internet and have earned both their popularity and (where it exists)
their revenue. But the end-to-end principle and the reliability of
distributed processing must have their day again, whenever some use of
the Internet becomes too important to leave up to any single entity.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/QPc-hsVUGf4/rss-never-blocks-you-or-goes-d.html
Better BBQ Through Chemistry -- food is the perfect ground for geek training: there are measurements, there's science, it's easy to know whether you've succeeded, and you can eat all but the worst of your failures. (via BoingBoing)
NoSQL (East) -- conference on East Coast for relationless databases.
Human Brain Processing Speed -- clocked at 60bits/second, according to this MIT Technology Review article. Their approach eventually led to Hick's Law, one of the few laws of experimental psychology. It states that the time it takes to make a choice is linearly related to the entropy of the possible alternatives. The results from various reaction-time experiments seem to show that this is the case. Although one byproduct of this approach is that the results are intimately linked to the type of experiment used to measure the reaction time. And that makes each study peculiarly vulnerable to the idiosyncrasies of the experimental approach. Today, Fermi Moscoso del Prado Martín from the Université de Provence in France proposes a new way to study reaction times by analyzing the entropy of their distribution, rather in the manner of thermodynamics. (via Hacker News)
Truly Social Data -- Data will only be truly social when you can work with it in the kinds of ways we work with information in the real, non-computational, world. In the real world we don’t ask for permission to have an opinion on something, to add to the ball of information surrounding a concept. Our needs don’t have to be anticipated by programmers. We can share information as we please. For example, nobody owns the concept of Barcelona. If I want to essentially “tag” Barcelona as being hot, or noisy, or beautiful, I just do it. I can keep my opinion private, I can share it with certain others, I can hold conflicting opinions, I can organize things in multiple ways at the same time and give things many names.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/OBEa_bIUyFo/four-short-links-26-august-200.html
Since my last post on the number of active Facebook users, the company once again doubled in Asia, adding more than 14 million active users over the last 12 weeks.
Through the latter part of last week, the company had over 266 million active users.
As the company becomes more mainstream in the U.S., the share of users under 25 years old continues to decline (it went from 45% to 37% in the last 12 weeks). Compare that to regions and countries where the company is growing fastest: users 25 years old or younger accounted for 58% of users in Asia, 54% in South America, and 60% in the Middle East/North Africa.
Over the last 12 weeks teens (13-17) was the second fastest-growing age group in Asia.
Details, including active users by country and age group, can be found below:
Active Facebook Users By Country & Region: August 2009View more documents from oreillymedia.
() We maintain a data mart that contains the number of active Facebook users (dating back to May-08), grouped using a variety of factors including age, country, and gender. Data for this post was through 8/14/2009.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/zRey7rHnEe0/facebook-is-younger-in-asia-and-the-middle-east.html
Guest blogger Carl Hewitt, Emeritus at MIT in the Electrical Engineering and Computer Science department, is known for his research on strongly paraconsistent logic, privacy-friendly client cloud computing, norms and commitments for organizational computing, and concurrent programming languages, models, and theories.
Aggregators (Google, Yahoo, Microsoft, Facebook, etc.) tend to believe that personal information is a valuable asset for several reasons. It is valuable to advertisers because it enables greater relevance for their ads. It is valuable to users because it can be used to enrich their lives. And it is valuable to aggregators because they can use personal information to make more money by selling (anonymous?) versions and by using it to bring together advertisers and customers. Recency and intimacy can add value to information. Current and recent information tends to be more relevant than older information. Intimate psychological, physiological, sociological, geographical, medical, etc. information can be used to personalize interactions.Intimate current personal information is also valuable for government security because it can be critical to taking security counter measures. Already in the UK, the previous two years of everyone's email, web browsing, and telephone calls are becoming available to government officials at varying levels of detail. For example, detectives will be required to consider accessing telephone and internet records during every investigation under new plans to increase police use of communications data.But that's only the beginning. As Jim Gray noted in "Distributed Computing Economics" (MSR-TR-2003-24) there is a growing imbalance between the computation power of billions of cores in aggregator datacenters and the relatively feeble fiber optic communications coming out of aggregator datacenters. This problem has now become so severe that Amazon has been forced to introduce a commercial service that lets users of their cloud import and export data through the post--as in, put it on storage devices and ship it by land, sea, or air. Soon even this stopgap will become impractical for government security agencies because whole shipping containers would have to be transferred--the functional equivalent of shipping large pieces of an aggregator datacenter. Consequently, to be effective, future government security software will have to be tightly integrated with aggregator datacenters. The most effective security measures will require aggregator datacenters to be heavily regulated, i.e., analogous to nuclear power plants.Semantic Integration, an emerging technological capability to bring together all kinds of information in a semantic engine, will greatly intensify all of the above issues (see "A historical perspective on developing foundations for privacy-friendly client cloud computing: The Paradigm Shift from 'Inconsistency Denial' to 'Semantic Integration' " ArXiv 0901.4934). The following kinds of information can be semantically integrated: calendars and to-do lists, email, SMS and Twitter archives, presence information (including physical, psychological and social), maps (including firms, points of interest, traffic, parking, and weather), events (including alerts and status), documents (including presentations, spreadsheets, proposals, job applications, health records, photos, videos, gift lists, memos, purchasing, contracts, articles), contacts (including social graphs and reputation) and search results (including rankings and ratings).Two critical technologies are the foundation of Semantic Integration: The first is lightly structured natural language interfaces that allow information to be easily found and organized. The second is many-core semantic engines (see "ActorScriptTM: Industrial strength integration of local and nonlocal concurrency for Client-cloud Computing""; ArXiv 0907.3330) that rapidly process information in ways that are tolerant of inconsistency (see "Common sense for concurrency and strong paraconsistency using unstratified inference and reflection" ArXiv 0812.4852).To be effective, government security Semantic Integration systems will need to be joined with those of aggregators. Thus Semantic Integration of personal information on aggregator datacenters will require additional government regulation of aggregators. Will government regulation prove toxic to the ability of aggregators to innovate?This is a future that we expect most readers would find distasteful. There is an alternative: A client cloud is a local cloud controlled by a client, e.g., a family cloud might consist of the cell phones, computers, security cameras, home entertainment centers, Wi-Fi access points, etc. of a family. Semantic Integration could be performed in clients' clouds so that clients by default store their information in cloud datacenters in a way that it can be decrypted only by using a client';s secret key.Semantic Integration using clients' clouds has some important advantages. Client responsiveness can be faster by not requiring communication with datacenters. Aggregator capital, operating and communication costs can be lower because Semantic Integration is performed in clients' clouds instead of aggregator datacenters. By performing Semantic Integration in clients' clouds, aggregators can make tons of more money than now by doing an even better job of matching up customers with merchants in a way that is more pleasing to both. Aggregators can provide software that runs in the clients' clouds (although it may have to be audited by 3rd parties). The aggregator's software can volunteer high level information to the aggregator's datacenters about the kind of merchant information that might be relevant. Within clients' clouds, the merchant information can then be tailored to the specific requirements of clients.For reasons above, an aggregator can do better by performing clients’ Semantic Integration using their clouds rather than relying entirely on the aggregator's cloud. And using clients’ clouds could lessen the degree of government regulation because the government would have to subpoena clients to obtain their most intimate personal information. If the information in an aggregator’s datacenters is sufficiently anonymous, then it would not become necessary for government security agencies to regulate them so heavily.The question is: "What are the aggregators going to do about intimate personal information?" If one of them initiates a project to develop a Semantic Integration product that operates in clients' clouds, then the others will rapidly follow suit.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/L-Sy37y26x0/is-intimate-personal-informati-1.html
Reboot Britain Video Archive -- video from the talks at Reboot Britain are online. The event also produced a essay set (PDF), CC-licensed. (via Paul Reynolds)
Revealing Errors -- Benjamin Mako Hill blog using computer errors as starting points for understanding how computers control the world around us. (via Dan Meyer)
New Microbe Strain Makes More Electricity, Faster -- University of Amherst researchers made current-generating bacteria work harder to live, and in five months had a strain that made an 8x larger current.
Netflix Culture -- readable slide deck which talks about the Netflix company culture. It's hard to read it and not nod in full agreement. (via joshua on Delicious)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/BBH6eCaWyeg/four-short-links-5-august-2009.html
Enabling Massively Parallel Mathematics Collaboration -- Jon Udell writes about Mike Adams whose WordPress plugin to grok LaTeX formatting of math has enabled a new scale of mathematics collaboration.
2845 Ways to Spin The Risk -- introduction to the ways in which our perception of risk (and numbers in general) can be distorted by how it is presented. (via titine on Twitter)
Logstalgia -- OpenGL app to visualize Apache log files.
4Store -- "scalable RDF storage". 4store was designed by Steve Harris and developed at Garlik to underpin their Semantic Web applications. It has been providing the base platform for around 3 years. At times holding and running queries over databases of 15GT, supporting a Web application used by thousands of people. (via joshua on Delicious)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/9gAiPB2-sYI/four-short-links-3-august-2009.html
Ignite Seattle 7 is being held a week from Monday, August 3rd, 2009 at the King Cat Theater. We have an all-star line up of some of the most ingenious, fascinating and inspiring geeks from across the world (or at least in or around Puget Sound).
Doors open at 7. Talks begin at 8:30. The opening contest will be a massive Rock-Paper-Scissors tournament. You join the team of the person you lose to until hundreds of people are on each team.
Here are the first 13 speakers listed in no particular order. We'll announce the final 5 next week.
Daniel Westreich (danielwestreich) - Causal inference is hard; or how I learned to stop worrying and love counterfactuals
The philosophical and practical problems of causal inference, and how to overcome these problems using randomized trials. With particular application to medical literature and epidemiology more generally.
Lee LeFever (leelefever) - Where Goldfish Come From
Everyone knows goldfish and koi, but very few have ever thought about where they come from - how they are bred, raised, transported, etc. I know these things like the back of my hand.
Dan Shapiro (danshapiro) - Making Benjamin Fly: Geeking out aero-style for about a hundred bucks
When I was a kid, RC flight meant spending thousands of dollars to put what was essentially a slightly-aerodynamicized lawnmower in the air. You spent thousands on engines and electronics and balsa, months building your plane, crashed it your first flight out, and then repeated. Over, and over, and over again. Enter lithium polymer batteries, rare earth magnets, miniaturized solid state inverters, 2.4 GHz spread spectrum frequency hopping transmitters and receivers. What do you get? I'll show you. And I'll show you how to get it up, for about one benjamin.
Mandy Sorensen (mandercrosby) - What To Do With 60 Minutes in Whale (and How I Learned to Use a Machete!)
Ever wondered what to do with a half-alive beached whale on a remote island in the Pacific?
Mehal Shah (mehals) - Fighting Dirty in Scrabble
Are you tired of your family thrashing you at Scrabble? Do you wince when someone brings out that red box at board game night? Are you ready to wipe the smug grin off the face of your significant other who pulls 7-letter words out of nowhere?
Elan Lee (elanlee) - I Wish I Was Taller
I filed a bug on my life with a major software company in Redmond.
Lauren Bricker (brickware) - Geek Generation
Don't call me a teacher, I'm more of a “Geek Generator.” I have kids (9 and 18), both who love computers and yes, they've already learned how to program. But apparently that wasn't enough for me. For the last two years I've been teaching computer science at a local private high school. It's incredibly interesting, rewarding, and yes, a lot of work. My goal with this talk is to generate more Geek Generators.
Willow Brugh (willowbl00) - Creating Communal Creative Space
The experience of building a maker space from scratch is certainly a project - I'll talk about my experience in doing so, what advice others have shared with me, and what spaces like this are already available in Seattle (and perhaps elsewhere on the West Coast).
Mónica Guzmán (moniguzman) - Addiction! Staying afloat in the age of the stream
Glued to email, your RSS reader or Twitter? Has your hand grown by 133 grams and the approximate weight of an iPhone? The Web is a stream, and it's easy to drown. Tips, tricks and cautionary tales from a reporter who swims the stream to stay on top of local news, but has learned the hard how easy it is to get carried away.
Gregory Heller (gregoryheller) - What Makes The Greenest Cab?
Green transportation is all the rage these days, especially hybrid vehicles. Popular wisdom may lead some, including civic leaders and politicians to believe that the greenest vehicle is a hybrid. NYC Mayor Mike Bloomberg has been fighting to Green the Yellow Cab fleet in that city by forcing all new cabs to be hybrids. The iconic NYC TaxiCab often sets the pace for the rest of the country's cabs. However would hybrids in NYC really make green cabs? And would the rest of the country's cab industries follow suit? The answer may surprise you.
Todd Sawicki (sawickipedia) - How I learned to Appreciate Dance Being Married to a Ballerina
Often times we see talks about how spouses deal with being married to geeks and startup jocks, now its time to turn the tables. This is a talk on what I've learned about ballet and how to appreciate it being married to a former professional ballerina. Hopefully you too will be able to tell the difference between a first and fifth position and a Plié vs. a Passé. Even a geek can learn to love classical dance.
Yoram Bauman - Principles of economics, translated
Translates for a lay audience the 10 principles of economics from Harvard professor Greg Mankiw's best-selling textbook.
Deepak Singh (mndoci) - Big Data and the networked future of science
New instruments, sensors, distributed scientific collaboration, informal publication channels = lots of data. How do we crunch it? How do we share it? How do we distribute it? This talk will dive into (a very very fast dive) into the challenges and solutions of the big science of today and tomorrow. Exascale anyone?
If you're not sure what Ignite is check out this promo video for Ignite Baltimore:
(Video courtesy of Think Again Media)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/DLhDzLX-JbQ/ignite-seattle-7-on-83-speaker.html
On Data Reconciliation Strategies and Their Impact on the Web of Data -- For years, I’ve been a fairly vocal advocate for the elegance and scalability of a-posteriori reconciliation via equivalence mappings as a superior mechanism (scale-wise) to a-priori reconciliation efforts
but this started to change very rapidly once I started working for Metaweb and saw first hand how much more effective a-priori reconciliation can be, even if drastically more expensive and limiting in the data acquisition front. (via straup on Delicious)
Java Spring's Biggus Dickus Effect -- Nonstop administrative debris as dadaist poetry. Écriture automatique of the programming office manager or his parrot. (via mattb on Delicious)
Arabic Blackberry Spyware -- update pushed out to Arabic Blackberries CC:ed all email to the authorities. A powerful case for multi-distro platforms, which reduces the size of the market captured with one distro is pwned like this.
NaCl - Networking and Cryptography Library -- open source high-level crypto library. NaCl (pronounced "salt") is a new easy-to-use high-speed software library for network communication, encryption, decryption, signatures, etc. NaCl's goal is to provide all of the core operations needed to build higher-level cryptographic tools. Of course, other libraries already exist for these core operations. NaCl advances the state of the art by improving security, by improving usability, and by improving speed. Creator of qmail is one of the developers. (via Simon Willison)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/aKW7cjNfs40/four-short-links-21-july-2009.html
News providers, like most content providers, are interested in having their content seen by as many people as possible. But unlike many news organizations, whose primary concern may be monetizing their content, National Public Radio is interested in turning it into a resource for people to use in new and novel ways as well. Daniel Jacobson is in charge making that content available to developers and end users in a wide variety of formats, and has been doing so using an Open API that NPR developed specifically for that purpose. Daniel will talk about how the project is going at OSCON, the O'Reilly Open Source Convention. Here's a preview of what he'll be talking about.James Turner: Can you start by explaining what NPR Digital Media is and what your role with it involves?
Daniel Jacobson: Sure. NPR is a radio organization, of course, and the Digital Media Group, of which I'm a part, handles, essentially as I describe it, everything that is publishable by NPR that does not go to a radio. So that includes the website, podcasts, API, mobile sites, HD radios, anything that has some sort of visual component to it. So Digital Media as a group is responsible for producing that content, producing all of those distribution channels, managing all of those relationships.
James Turner: And what is your particular role there?
Daniel Jacobson: I manage the application development team that is responsible for all the functional aspects of all of the systems, which includes our CMS, all of the templating engines for the website, for the API, for the podcasts, all of the engines that drive that.
James Turner: Now NPR is an organization that consists of a lot of member stations kind of flying in close formation. What's your relationship with the content producers? To what extent do they have their own stuff, and to what extent do you work together?
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/rtxVsY1TBTI/how-npr-is-embracing-open-sour.html
IDEO's Human Centered Design Toolkit -- methodology and toolkit for inspiring new solutions to difficult challenges within communities of need. Full PDF of manual and cards available for free download.
Bentham and the Privacy of the Grave -- [M]uch of what Bentham meant to address in the context of his Panoptic structures we now take for granted. In Bentham’s lifetime, Parliamentary deliberations were confidential. Bentham’s arguments forced them into the sunlight. Legal decisions and statute books were accessible only to lawyers and judges. Bentham’s arguments led to codification of the law, and increasingly accessible legal rules. Bentham was far ahead of his time — the first modern information theorist. The idea that all actions of government would be presumptively available for public review did not become part of U.S. law until the passage of the Freedom of Information Act (FOIA) in 1967. As we speak, it appears the English parliament is only now learning Bentham’s message about publicity. Bentham was an early transparency advocate, economist, and character. I first read of him in the excellent A Brief History of Economics: Artful Approaches to the Dismal Science. (via carlmalamud on Twitter)
Curated Twitter Feed for Projecting Over Speakers -- Guardian developed it for their "Activate Summit" and it's since been used in two other events. They've open sourced it.
Android Market Problems -- take heed, all ye who would build "the iPhone App Store of ...", it's not easy to deliver a great customer experience.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/6P9Vw50Wk9E/four-short-links-13-july-2009.html
The World Economic Forum started a
research project
at Davos 2009 concerning cloud computing, which they broadly define to
include all kinds of remote services, from Software as a Service to
virtual machines.I was asked to provide some ideas on the implications of cloud
computing for business as well as its future operating environment. To
allow my colleagues and the O'Reilly community to help define the
issues and provide references, I've put up a
discussion forum as a wiki.
Anyone with relevant and valid ideas can suggest points. I don't even
mind people listing their businesses and information sources, so long
as the information is relevant and is directed toward the larger
educational goal of the wiki.I'm holding a phone call with someone from WEF on Thursday,
July 16 where I'll present my views, augmented by your suggestions.Here are the categories of the notes I used to seed the wiki.
Benefits and drawbacks for potential clients
Benefits and drawbacks of offering software as a service or using a development environment
Access
Resilience
Portability
Environmental implications
Software freedom
Government use
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/ne9E5lkMVhE/cloud-computing-perspectives-a.html
As some of you may know, I got my undergraduate degree in Greek and Latin Classics. So when Forbes asked me to do an interview on the subject of how my Classical education had affected my business career, I agreed. The result, part of a special report called Power, Ambition, Glory, used only a small part of the interview I provided, so I decided to publish the entire interview here. Questions in bold were provided by Forbes in an email interview:
1. Tell us about a time when lessons learned from the ancients contributed to your success.
I love this question. As John Cowper Powys noted in The Meaning of Culture, culture (vs. mere education) is how you put what you've learned to work in your own life, seeing the world around you more deeply because of the historical, literary, artistic and philosophical resonances that current experiences evoke. Classical stories come often to my mind, and provide guides to action (much as Plutarch intended his histories of famous men to be guides to morality and action). The classics are part of my mental toolset, the context I think with. So rather than giving you a single example, let me give you a potpourri.
The unconscious often knows more than the conscious mind. I believe this is behind what Socrates referred to as his inner "daimon" or guiding spirit. He had developed the skill of listening to that inner spirit. I have tried to develop that same skill. It often means not getting stuck in your fixed ideas, but recognizing when you need more information, and putting yourself into a receptive mode so that you can see the world afresh.
This skill has helped me to reframe big ideas in the computer industry, including creating the first advertising on the world wide web, bringing the group together that gave open source software its name, and framing the idea that "Web 2.0" or the "internet as platform" is really about building systems that harness collective intelligence, and get better the more people use them. Socrates is my constant companions (along with others, from Lao Tzu to Alfred Korzybski to George Simon, who taught me how to listen to my inner daimon.)
I believe that I've consistently been able to spot emerging trends because I don't think with what psychologist
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/0BbW5VD8AnQ/benefits-classical-education.html
Harvard Study Finds Weaker Copyright Protection Has Benefited Society (Michael Geist) -- Given the increase in artistic production along with the greater public access conclude that "weaker copyright protection, it seems, has benefited society." This is consistent with the authors' view that weaker copyright is "uambiguously desirable if it does not lessen the incentives of artists and entertainment companies to produce new works." (read the original paper)
Using Public Data for Good With the Power of YQL -- The first part is a new batch of YQL tables providing data on the U.S. government, earthquake data, and the non-profit micro-lender Kiva. The second part is an incredibly easy way to render YQL queries on websites. After all, what good is data that no one can see?
GeoSPARQL -- RDF meets geo goodness. SELECT ?s ?p ?o WHERE { ?s gn:name "Dallas" . ?s ?p ?o } (via the geowanking mailing list)
How To Be Happy in Business -- this Venn diagram makes me happy. (via Ned Batchedler)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/2YRwrV4GwNY/four-short-links-18-june-2009.html
I love the idea of Fire Eagle, Yahoo's under-supported location-brokering service. However until recently I found myself unable to use it. I had no mobile service that I could consistently rely on to update Fire Eagle.
Enter Clarke. Clarke (named after Arthur C. Clarke) is a small tool that runs in the background on my Mac. It updates my location on FireEagle every 5 minutes. It triangulates me via Skyhook (the same location-service that is used on the iPhone). Like all FireEagle apps Clarke used OAuth to gain access to my account (developer Tom Taylor used oauthconsumer). Tom has released the Clarke code on Github.
If you don't have a Mac or don't wish to run Clarke for some other reason these Fire Eagle updater might do the trick for you. Fire Eagle Updater Add-on for Firefox has a very descriptive name. This Firefox extension was developed by Yahoo! and requires the Geode extension. Unfortunately, there doesn't appear to be a similar updater for Windows.
The reason I bring up Clarke is not that you can update FireEagle; there are plenty of apps (79 in the gallery) that work with the service. Instead, it's interesting because of its continuous background updates. I don't have to think about it and can leave it running all the time, which means it gets forgotten.
When I setup a continuous location-updater I have to think about the actions of my future self. How can I guard against him? What if there are places or times that I do not want revealed (shopping for a birthday present, my home, etc.)? I take care of this by usually only sharing my neighborhood (it's useful for both locals and non-locals without getting too specific). However, in the future I'd like time-based rules (share specific places after 5PM), white lists (always share bars or when I am in San Francisco) and black-lists (never share my home). Services like Clarke and Fire Eagle and Latitude will need to add these safeguards before they get wide spread adoption.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/jl54Xc9ZuMg/clarke-and-the-continuous-loca.html
Ignite! is coming to San Jose on Monday June 22, 2009 at 8:00 pm, attached to the Velocity Conference. Admission is free, open to all, and there will be a cash bar.The deadline for talks is May 11th, so submit your talks now!As with all Ignites each speaker will only get 20 slides that each auto-advance every 15 seconds for a total of five minutes. We'll be looking for fun geek topics like hacks, how-to's, and insights. (Talks don't have to be Velocity-related!) If you're not sure what an Ignite talk looks like check out the Ignite Show.You can RSVP for the event on Upcoming or Facebook.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/xT9a5MOlDJg/ignite-comes-to-san-jose-june.html
I've been reading a lot of philosophy lately -- Kierkegaard and Dawkins, Lewis, Hume, Calvin and Augustine, you name it -- for a class I'm taking, as well as for my own enjoyment. One of the interesting things about philosophy is that it's a discipline that takes the understanding of understanding seriously. As a teacher, that's fascinating to me; has education -- specifically, the way we in 2009 are trying to educate -- really examined what knowledge is? Have educational systems considered what the wealth of literature says about knowledge, and responded to it responsibly?[A few important insertions: I'm not supposing that to respond to philosophical ideas about knowledge, education needs to change. I am suggesting, though, that a responsible response entails understanding the arguments, and either adhering to them, or forming a sound counter-argument to explain abandoning them.]Two theses in particular caught my attention. First, Thomas Reid makes this astounding statement (cited from Thomas Reid's Inquiry and Essays (1975), p. 275):Another first principle is--That the natural faculties, by which we distinguish truth from error, are not fallacious.What does this say? It says that our natural senses don't tend to fail us. Now, I know, many of you are freaking out over this quote, especially in light of particle physics or molecular biology. And we can argue that over a good cup of coffee. But I will suggest that Reid is right in the macro-world. I see a piano falling, I rightly assume several things:1. It is indeed falling, and I am not instead rushing up toward it.2. That piano is dangerous to be under, given that it's falling.There are plenty of other observations, but you get the idea.So why does this matter? Well, how much do we allow the learner's senses (and by senses, I don't mean "ears listening to 90 minutes of lecture") engage in a typical learning environment? How much do we allow the natural faculties to assert themselves, create knowledge, and then refine and provide context for the knowledge already gained?I think it's an important question. I think Dan Meyer is a master at this (check out his blog for some amazing examples of using pictures to stimulate learning). How are you doing this? What effect on education would an increased (as in, significantly more than you're currently doing) amount of sensory learning create?Let me know what you think. Oh, and as for my other philosophical quote that I think is important? More on that later this week...
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/763K0flGhug/when-do-your-beliefs-become-kn.html
In my previous post, I noted that despite the large decline in total number of job postings, the number Hadoop/MapReduce job postings increased by 49%. What is the current state of the online job market? The financial crisis that began in the Fall of 2008 has had a lasting negative effect on the U.S. online job market. Since late 2008, there have been significantly less jobs posted online.
Using data from SimplyHired and a few charts, I'll quickly highlight the impact of the global economic crisis on the U.S. online job market. To quantify the sudden drop in U.S. online job postings, I calculated the average number of job posts per day:
The number of posts declined 49% from Jan/May 2008 to Jan/May 2009. While there has been a downward trend since April 2008, the financial crisis in September 2008 marked the start of even larger reductions. In particular, the relatively small number of job postings in Nov/Dec 2008 has carried over into the first five months of 2009. The sharp seasonal rebound that occurs in Jan/Feb of each year, was practically non-existent in 2009. While some forecasters are seeing signs of a recovery, at least through the first five months of 2009, we haven't detected "green shoots" in the U.S. online job market.
Looking at job postings by location, the Top 20 states (in terms of # of postings from Jan/May 2009), suffered losses ranging from 38% (MD,VA) to 58% (MN) fewer postings per day:
Declines weren't confined to states with large numbers of job postings, or certain regions of the country. Every state suffered large drops: the "best-performing" state (OK) saw a 36% decline in number of job postings per day.
() In partnership with SimplyHired and Greenplum, we maintain a data warehouse that contains most U.S. online job postings dating back to mid-2005.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/5Wcj02SEdN8/the-economic-crisis-and-the-us-job-market.html
Some people never grow up. Some people wait to have children so that they can become kids again. When Hillel Coopermans's young ones were four he began his family's Lego collection. This led to whole rooms being devoted to the hobby, to eBay auctions, and Leog conventions. In this week's Ignite Show Hillel takes us through the Lego Underground. He delves into Lego CAD and custom Lego pieces. If you need more Lego check out Wired's gallery of Lego projects for this weekend's Maker Faire.
You can also get the Ignite Show on iTunes.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/89AI0iiQl6U/ignite-show-hillel-cooperman-o.html
After the press conference following this morning's keynotes, I was part of a small group conversation with Lars Rasmussen, head of the Google Wave team. He told the story of how they pitched Sergey Brin on the Wave project. "We'd worked on our message," he said, "and we boiled it down to this: 'We think we have an idea that will have a bigger impact on email than Google Maps had on maps.'" Sergey bought off on the idea. 'Nuff said.
Lars pointed out that he and Jens actually had enough "accrued" 20% time that they could arguably have done the project anyway. "We hadn't taken any 20% time since we started. So we had about eight months each saved up. You aren't really able to save up that much of it though, but we were prepared to make that argument if we needed to."
Lars had already moved to Sydney, and made the case that Wave could best be created there, where the team could operate as a kind of independent startup. Jens moved over, and they built the first prototype over nine months with a team of five, during 2007. Since then, the team has grown to about 100.
Judging from the number of people in the technical sessions on Wave at Google I/O, that development team just got a lot larger!
Lars also mentioned an interesting point about how developers can get their work noticed in Wave: you share extensions simply by using them in a wave. They can simply be installed from there. This will provide a viral vector for adoption of new extensions.
Lars pointed out that there is one way that Google's internally-developed extensions have "more power" than external extensions: some of them come pre-installed for all waves. They haven't figured out yet the right way to give that kind of access to third parties. But they are definitely thinking about it - and how automated trust metrics (e.g. how many people are using an extension) might be used to promote the work of external developers.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/Guhr_agkh7U/google-wave-the-early-days.html
This post is part three of a series raising questions about the mass adoption of social technologies. Here are links to part one and two. These posts will be opened to live discussion in an upcoming webcast on May 27. (special guest to be announced shortly)In 1785 utilitarian philosopher Jeremy Bentham proposed architectural plans for the Panopticon, a prison Bentham described as "a new mode of obtaining power of mind over mind, in a quantity hitherto without example." Its method was a circular grid of surveillance; the jailors housed in a central tower being provided a 360-degree view of the imprisoned. Prisoners would not be able to tell when a jailor was actually watching or not. The premise ran that under the possibility of total surveillance (you could be being observed at any moment of the waking day) the prisoners would self-regulate their behavior to conform to prison norms. The perverse genius of the Panopticon was that even the jailor existed within this grid of surveillance; he could be viewed at any time (without knowing) by a still higher authority within the central tower - so the circle was complete, the surveillance - and thus conformance to authority - total.
In 1811 the King refused to authorize the sale of land for the purpose and Bentham was left frustrated in his vision to build the Panopticon. But the concept endured - not just as a literal architecture for controlling physical subjects (there are many Panopticons that now bear Bentham’s stamp) - but as a metaphor for understanding the function of power in modern times. French philosopher Michel Foucault dedicated a whole section of his book Discipline and Punish to the significance of the Panopticon. His take was essentially this: The same mechanism at work in the Panopticon - making subjects totally visible to authority - leads to those subjects internalizing the norms of power. In Foucault’s words “
the major effect of the Panopticon; to induce in the inmate a state of conscious and permanent visibility that assures the automatic functioning of power. So to arrange things that the surveillance is permanent in its effects, even if it is discontinuous in its action; that the perfection of power should tend to render its actual exercise unnecessary” In short, under the possibility of total surveillance the inmate becomes self regulating. The social technologies we see in use today are fundamentally panoptical - the architecture of participation is inherently an architecture of surveillance.In the age of social networks we find ourselves coming under a vast grid of surveillance - of permanent visibility. The routine self-reporting of what we are doing, reading, thinking via status updates makes our every action and location visible to the crowd. This visibility has a normative effect on behavior (in other words we conform our behavior and/or our speech about that behavior when we know we are being observed). In many cases we are opting into automated reporting structures (Google Lattitude, Loopt etc.) that detail our location at any given point in time. We are doing this in exchange for small conveniences (finding local sushi more quickly, gaining “ambient intimacy”) without ever considering the bargain that we are striking. In short, we are creating the ultimate Panopticon - with our data centrally housed in the cloud (see previous post on the Captivity of the Commons) - our every movement, and up-to-the-minute status is a matter of public record. In the same way that networked communications move us from a one to many broadcast model to a many to many - so we are seeing the move to a many-to-many surveillance model. A global community of voyeurs ceaselessly confessing to "What are you doing? (Twitter) or "What's on your mind? (Facebook)
Captivity of the Commons focused on the risks corporate ownership of personal data. This post is concerned with how, as individuals, we have grown comfortable giving our information away; how our sense of privacy is changing under the small conveniences that disclosure brings. How our identity changes as an effect of constant self-disclosure. Many previous comments have rightly noted that privacy is often cultural -- if you don't expect it - there is no such thing as an infringement. Yet it is important to reckon with the changes we see occurring around us and argue what kind of a culture we wish to create (or contribute to).
Jacques Ellul’s book, Propaganda, had a thesis that was at once startling and obvious: Propaganda’s end goal is not to change your mind at any one point in time - but to create a changeable mind. Thus when invoked at the necessary time - humans could be manipulated into action. In the U.S. this language was expressed by catchphrases like, “communism in our backyard,” “enemies of freedom” or the current manufactured hysteria about Obama as a “socialist”. Similarly the significance of status updates and location based services may not lie in the individual disclosure but in the significance of a culture that has become accustomed to constant disclosure.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/XoHrXWOSy-c/the-digital-panopticon.html
Ever since Twitter started suggesting accounts to new users, it was clear that those on the suggested users list were gaining thousands of followers. Setting aside the fact that number of followers is a poor gauge of influence (see our Twitter report for details), I wanted to know how many followers a suggested account gains by appearing on the list.
I took the set of accounts that were added to the suggested users list during the last two months, recorded their number of followers the day before they made the list (Initial # of followers), and tracked what happened a week, 2 weeks, and a month later. From an initial set of just over a hundred accounts, I was able to gather sufficient data (using Twitterholic and Twittercounter) on 80+ suggested users.
On average, a suggested user gained about 53,000 followers a week after first appearing on the list. In comparison, on average the same users added "only" 1,900 followers the week prior to their appearance on the suggested users list. The effect persisted‡ over time: on average, the same users gained about 198,000 followers one month after being on the list. (I captured the persistent increase in followers visually, using statistical distributions.)
The effect is even more pronounced for accounts with large numbers of initial followers. In the table below, I compared the bottom quartile (at most 3,000 initial followers) with the top quartile (at least 25,000 initial followers):
Two weeks after appearing on the list, the bottom quartile gained about 93,000 followers. In contrast, the top quartile gained 105,000 followers over the same 2-week period.
The above results quantify what is common knowledge: becoming a suggested account is a quick way to boost one's number of followers. But there are only 200+ accounts on the list of suggested users, what about the rest of Twitterdom? As our resident Twitter expert Sarah Milstein points out, there are techniques that let users extend their reach (apart from obsessing over how to increase their number of followers). Check the Twitter book for specific suggestions.
() In the absence of a control group, I'm left with observing how many followers the suggested accounts added the weeks before and after making the list. As expected, I get similar results when I use the MEAN instead of the MEDIAN. Having looked through enough suggested accounts, I'm confident a formal study will lead to the same conclusion: becoming a suggested user translates to thousands of additional new followers.
(‡) Once an account appears on the list of suggested users, it tends to remain there. Regardless, the few who were taken off seem to enjoy similar boosts in number of followers.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/S_LzjcHaEq0/twitter-suggested-users-list.html
There's a long-time debate between those who advocate for semantic markup, and those who believe that machine learning will eventually get us to the holy grail of a Semantic Web, one in which computer programs actually understand the meaning of what they see and read. Google has of course been the great proof point of the power of machine learning algorithms.
Earlier this week, Google made a nod to the other side of the debate, introducing a feature that they call "Rich Snippets." Basically, if you mark up pages with certain microformats ( and soon, with RDFa), Google will take this data into account, and will provide enhanced snippets in the search results. Supported microformats in the first release include those for people and for reviews.
So, for example, consider the snippet for the Yelp review page on the Slanted Door restaurant in San Francisco:
The snippet is enhanced to show the number of reviews and the average star rating, with a snippet actually taken from one of the reviews. By contrast, the Citysearch results for the same restaurant are much less compelling:
(Yelp is one of Google's partners in the rollout of Rich Snippets; Google hopes that others will follow their lead in using enhanced markup, enabling this feature.)
Rich snippets could be a turning point for the Semantic Web, since, for the first time, they create a powerful economic motivation for semantic markup. Google has told us that rich snippets significantly enhance click-through rates. That means that anyone who has been doing SEO is now going to have to add microformats and RDFa to their toolkit.
Historically, the biggest block to the Semantic Web has been the lack of a killer app that would drive widespread adoption. There was always a bit of a chicken-and-egg problem, in which users would need to do a lot of work to mark up the data for the benefit of others before getting much of a payoff themselves. But as Dan Bricklin remarked so insightfully in his 2000 paper on Napster, The Cornucopia of the Commons, the most powerful online dynamics are released not by appeals to volunteerism, but by self-interest:
What we see here is that increasing the value of the database by adding more information is a natural by-product of using the tool for your own benefit. No altruistic sharing motives need be present...
(Aside: @akumar, this is the answer to your question on Twitter about why in writing up this announcement we didn't make more of Yahoo!'s prior support for microformats in searchmonkey. You guys did pioneering work, but Google has the market power to actually get people to pay attention.)
What I also find interesting about the announcement is the blurring line between machine learning and semantic markup.
Machine learning isn't just brute force analysis of unstructured data. In fact, while Google is famous as a machine-learning company, their initial breakthrough with pagerank was based on the realization that there was hidden metadata in the link structure of the web that could be used to improve search results. It was precisely their departure from previous brute force methods that gave them some of their initial success. Since then, they have been diligent in developing countless other algorithms based on regular features of the data, and in particular regular associations between data sets that routinely appear together - implied metadata, so to speak.
So, for example, people are associated with addresses, with dates, with companies, with other people, with documents, with pictures and videos. Those associations may be made explicitly, via tags or true structured markup, but given a large enough data set, they can be extracted automatically. Jeff Jonas calls this process "context accumulation." It's the way that our own brains operate: over time, we make associations between parallel data streams, each of which informs us about the other. Semantic labeling (via language) is only one of many of those data streams. We may see someone and not remember their name; we may remember the name but not the face that goes with it. We might connect the two given the additional information that we met at such and such conference three years ago.
Google is in the business of making these associations, finding pages that are about the same thing, and they use every available handle to help them do it. Seen in this way, SEO is already a kind of semantic markup, in which self-interested humans try to add information to pages to enhance their discoverability and ranking by Google. What the Rich Snippets announcement does is tell webmasters and SEO professionals a new way to add structure to their markup.
The problem with explicit metadata like this is that it's liable to gaming. But more dangerously, it generally only captures what we already know. By contrast, implicit metadata can surprise us, giving us new insight into the world. Consider Flickr's maps created by geotagged photos, which show the real boundaries of where people go in cities and what they do there. Here, the metadata may be added explicitly by humans, but it is increasingly added automatically by the camera itself. (The most powerful architecture of participation is one in which data is provided by default, without the user even knowing he or she is doing it.)
Google's Flu Trends is another great example. By mining its search database (what John Battelle calls "the database of intentions") for searches about flu symptoms, Google is able to generate maps of likely clusters of infection. Or look at Jer Thorp's fascinating project announced just the other day, Just Landed: Processing, Twitter, MetaCarta & Hidden Data. Jer simulated the possible spread of swine flu built by extracting the string "Just landed in..." from Twitter. Since Twitter profiles include a location, and the object of the phrase above is also likely to be a location, he was able to create the following visualization of travel patterns:
Just Landed - Test Render (4 hrs) from blprnt on Vimeo.
This is where the rubber meets the road of collective intelligence. I'm a big fan of structured markup, but I remain convinced that even more important is to discover new metadata that is produced, as Wallace Stevens so memorably said, "merely in living as and where we live."
P.S. There's some small irony that in its first steps towards requesting explicit structured data from webmasters, Google is specifying the vocabularies that can be used for its Rich Snippets rather than mining the structured data formats that already exist on the web. It would be more "googlish" (in the machine learning sense I've outlined above) to recognize and use them all, rather than asking webmasters to adopt a new format developed by Google. There's an interesting debate about this irony over on Ian Davis' blog. I expect there to be a lot more debate in the weeks to come.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/EMbD9S_WtSU/google-rich-snippets-semantic-web.html
In a few weeks, the Facebook application platform will mark its second anniversary. While it garnered lots of press coverage in the months after it launched, the arrival of the iTunes app store shifted attention away from Facebook's vibrant ecosystem. The media glow is understandable: among other things, the younger iTunes platform is adding apps at a much faster rate than Facebook or Myspace.
Games comprise a sizable chunk of app revenues in all three platforms and recent stories suggest that 2009 has been a great year for developers. The substantial revenue generated by popular Facebook (and Myspace) apps has been the subject of articles in VentureBeat, TechCrunch, and Inside Facebook. There have also been recent estimates for the revenue generated by iPhone apps (see here and here). Game developers in particular are benefiting from having a multitude of platforms: Games are the largest iTunes category, and the second largest category in both Facebook and Myspace. In addition, 4 of the top 10 most successful Facebook app providers are Game developers.
With the 2-year anniversary of Facebook's platform a few weeks away, I've put together an overview of the current state of Facebook and Myspace apps. At a time when iTunes attracts most of the media coverage, the Facebook & Myspace (particularly Facebook) platforms are quietly producing apps and developers with impressive usage numbers. Over the past week, there were more than 29,000 app sellers on the Facebook platform. Over 2% of those sellers released apps with at least 100,000 combined active users. Close to 700 different Facebook apps had at least 100,000 active users over the last month. Over the past month, 49,000 Facebook apps had at least 50 active users:
Usage numbers are important because advertising is a popular source of revenue for many Facebook, Myspace, and iPhone apps. Some Facebook and Myspace apps (especially Games) also rely on points systems and virtual items to generate revenue -- both of which benefit from high usage numbers. More details, including a list of the top developers and usage stats, in the slides below.
[ Related Radar Posts: I recently released an overview of iTunes app sellers and a summary of trends in Facebook Demographics. ]
Facebook and Myspace App Platforms: A Brief UpdateView more presentations from oreillymedia.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/T2q0fwBfH2M/facebook-app-platform-2-year-anniversary.html
Baby nerds, evil URL shorteners, reasoned discussion, and the Government straps its Web 2.0 on:
Books for Wee Nerds -- Forget Pat the Bunny -- your baby wants to Pat Schrodinger's Kitty! Help baby search for subatomic particles and explore the universe. (via Tim's tweets)
On URL Shorteners -- Joshua Schachter and Maciej Ceglowski on the downsides of URL shortening services like bit.ly et al.
Mending The Bitter Absence of Reasoned Technical Discussion (Alex Payne) -- We’ve come to accept that trying to have a reasonable discussion on the Internet is like insert any number of increasingly offensive metaphors here. Usenet, IRC, forums, blogs, and now media like Twitter have all been black-marked as houses unfit for reason to dwell within. And so we roll our eyes, sigh, and quietly accept the idiocy, the opportunism, and the utter disrespect for our peers and ourselves that is technical discussion on the Internet. This need not be the case. It is possible to have a reasoned technical discussion on the Internet. People do it every day, particularly in smaller online communities where social norms are easier to enforce. We can do it. (via SarahM
GSA signs agreements with Web 2.0 providers -- Flickr, YouTube, Vimeo, and blip.tv get agreements that make it legal for federal agencies to use those tools. Followup to my earlier cite of roadblocks to Web 2.0 tools for government use. (via Fiona's delicious links)
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/XuSpzWP6W8A/four-short-links-6-apr-2009.html
As I've written here recently, we've got some amazing sessions scheduled for Web2Open--the free unconference hosted by Web 2.0 Expo in SF this week. One that I'm particularly excited about is a new experiment, "Practice Your Customer Pitch."We're bringing in five startups who will get two minutes each to give their customer pitch (not their VC pitch), as if meeting a potential customer at a cocktail party (i.e., no slides but OK to drink if you want). To give them feedback, we've assembled a top-notch panel of serial entrepreneurs and marketing experts. It's not a competition, so there's no judging or ranking—just discussion among the entrepreneurs, panelists and other session attendees. We're trying this idea for the first time, so who knows how it will go? But in the entrepreneurial spirit, we've mitigated our risks: even if the format doesn't sing, the session can only be a hit given the participants. (Thanks to Sean O'Malley for helping us connect with a lot of these folks.)The rather impressive panel:
*Rashmi Sinha, moderator. SlideShare CEO
*Robert Acker, panelist. LiveSpot CEO
*Michael Cerda, panelist. cc:Betty CEO
*Nilofer Merchant, panelist. Rubicon Consulting CEOThe smart startups:
*CrowdVine, social networks for conferences
*dbTwang, Dogster for guitars
*Doodle, online scheduling magic
*Maestro Market, a Web 2.0 speakers' bureau
*Magoosh, customized test-prepThe session is on Weds, April 1 from 10:50 - 11:40a. If you still need a free pass for Web2Open, you can register using the code websf09opn. There's more general event info on the Open website.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/6H1xeheJrRk/web2open-an-exciting-experimen.html
Eric Ries, a serial entrepreneur, most recently was co-founder and Chief Technology Officer of IMVU, his third
startup. He's the author of the blog Lessons Learned, co-author of several books including The Black Art of Java Game Programming (Waite Group Press, 1996), and a Venture Advisor at Kleiner Perkins Caufield & Byers.
He. In 2007, BusinessWeek named Ries one of the Best Young Entrepreneurs of Tech. He'll be presenting on "The Lean Startup: a Disciplined Approach to Imagining, Designing, and Building New Products" at Web 2.0 Expo.
We're living in a time of renewed possibility for startups. Major trends - from the pain of the economic crisis to the disruption of web 2.0 - are breaking the old models and paving the way for a new breed of company. I call it the Lean Startup.
The Lean Startup is a disciplined approach to building companies that matter. It's designed to dramatically reduce the risk associated with bringing a new product to market by building the company from the ground up for rapid iteration and learning. It requires dramatically less capital than older models, and can find profitability sooner. Most importantly, it breaks down the artificial dichotomy between pursuing the company’s vision and creating profitable value. Instead, it harnesses the power of the market in support of the company’s long-term mission.
Tim O'Reilly has recently been advocating that as an industry we focus on building stuff that matters. In response, I want to try and present a way of building startups that can realize that dream. In particular, he as articulated three principles:
(1) Work on something that matters to you more than money,
(2) Create more value than you capture, and
(3) Take the long view.
Given the hype and easy credit that has been the hallmark of technology startups the past few years, it's been too easy for them to be unclear about whether they are really creating more value or just spending money to create the appearance of success. The lean startup approach tackles this problem from the very beginning of a startup's life. My experience is that startups need to be built from the ground up for learning about customers and what they will pay for. That means an obsessive focus on finding out "is our company vision really the path to a brave new world, or just a delusion?"
Read the stories of successful startups and, if the founders are willing to be honest, you will see this pattern over and over again. They started out as digital cash for PDAs, but evolved into online payments for eBay. They started building BASIC interpreters, but evolved into the world's largest operating systems monopoly. They were shocked to discover their online games company was actually a photo-sharing site.
Each of these companies were fortunate to have enough time, resources, and patience to endure the multiple iterations it took to find a successful product and market. The premise of the lean startup is simple: if we can reduce the time between these major iterations, we can increase the odds of success.
And here's where working on something that matters to you more than money is critical. When you're committed to something larger than yourself, every minute counts. Hype and transient success won't keep you going. But the simple process of finding out whether or not your vision is right will. Because people who are dedicated to the truth are more likely to fail fast, learn, and try again.
This is one core tenet of the lean startup approach, called customer development. It has its roots in previous eras of startups (you can read more in the original customer development book, The Four Steps to the Epiphany), but changes in the industry are making it possible to iterate much faster than ever before.
There was a time when many technologies required proprietary licenses, big company permission, or custom deal-making ability to access. Those days are rapidly coming to a close. Most technology startups now have access to a staggering array of high-leverage technologies: free and open source software (and, increasingly, hardware), the data-driven services of web 2.0, user-generated content, and cloud computing, just to name a few. What all of these trends have in common is the increased leverage that development teams enjoy, meaning that for every ounce of effort they expend in building product, they take advantage of the efforts of thousands or even millions of others.
There's no need to tell Radar readers that these technology trends make it cheaper to make new products. What's striking to me is that they enable teams to make new products faster. It's the speed with which companies can move through product iterations that will define this new era. Those that can experiment rapidly will be more likely to uncover what customers truly want; those that take advantage of these high-leverage technologies will be able to experiment the fastest.
Another way smart startups can work faster is to adopt agile product development practices. In traditional waterfall development, which assumes a top-down plan and stable development towards a well-defined outcome, a large number of software projects fail outright (and the lucky ones come in way over budget and way late). The rise of methodologies like Extreme Programming and Scrum has enabled teams to spend more time focused on focused on building products customers actually want and less time engaged in fruitless practices like writing documentation nobody reads or revising specs nobody adheres to.
Still, agile is not always well adapted to the startup experience. That's not entirely surprising, because most agile methodologies have their roots in big companies. They are specifically designed to build products in situations where the problem is known but the solution is unknown. Thus, they engage in rapid communication between the engineers and an authoritative in-house customer or product owner, who can give them fast resolution on feature decisions as they come up. This is a huge improvement over ever-more-detailed specification documents. But startups routinely face the problem that they don't even know what problem they are trying to solve.
The Lean Startup takes agile practices and evolves them for use in a startup. The net result is a focus on experimentation and extremely rapid deployment. At IMVU, my most recent startup, we built the systems that allowed us to deploy code to customers fifty times every day. When releases are measured in minutes, not months, you can build a company culture designed to avoid the biggest waste of all: building product nobody wants.
At IMVU, we shipped a product in just a few months. It was terrible. But we decided to charge for it anyway, iterating our way to a freemium business model that brings in more than a million dollars a month. In the early days, though, nobody was buying. It took months of constantly shipping features, measuring the results, and trying again before we realized what was wrong. Although we were able to get a few users to try the product for free, that wasn't good enough. We wanted to validate the riskiest part of our business plan: that we could get people to pay real money for virtual clothes.
We were always convinced that the next feature we were about to ship would be "the big one" that would fix the product and help us make our paltry monthly revenue targets - only $300 a month in those early days. But these dreams of the instant fix never materialized. No one major release solved the problem, because the problem wasn't a lack of features.
We eventually realized that our initial product concept, which had seemed so brilliant at the whiteboard, was fundamentally flawed. But because we took a disciplined approach to learning we were able to find out before it was too late. Because we had tried every variation of features, had measured the behavior of the people we were bringing in, and were committed to a revenue plan, we were forced to change direction. It was painful, but if we hadn't done it, we would never have been able to chart a course that led to our eventual success.
In fact, it wasn't the risky part of our original vision that had to change. It turns out that people really will pay good money for virtual goods. By discovering the other problems with our concept early, we were able to preserve our larger long-term vision. And, most importantly, we were then able to prove that it could work.
The Lean Startup is a vision for how startups could be built differently. Instead of focusing on hype and mega-growth, we can focus on building companies that serve customers in a fundamental way. That's what matters.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/-99hp57FlAw/lean-startup.html
I love this. Two university researchers are asking real world coders how they code. They want to learn whether the theory taught in software design courses is actually used in the real world. They've built a short (20-question) survey that takes less than ten minutes to complete, and will open source the data once it's all in. I've sent in my answers and I hope all the programmers reading this do so too. One of the researchers was my lecturer for Algorithms, which must have been punishment for his sins in a past life. The other influential prof from my degree was one of the authors of Notes on Postmodern Programming, and I hope my work mixes the unexpected and the pragmatic as well as they have.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/5uSAlazBp2Q/software-engineering-folklore.html
Guest blogger Dylan Field is an intern at O'Reilly and Senior at Technology High School in Rohnert Park, CA, where he is a member of the FIRST Robotics team, Dylan is especially interested in Computer Science, Mathematics,and Statistics.
In his "Web Meets World" talk at the Web 2.0 Expo in New York last September, Tim O'Reilly described where he saw the web heading. "The next stage of Web 2.0 is going to be driven by sensors," he said. "We are moving out of the world in which people typing on keyboards are going to be driving collective intelligence applications."
Like all transitions, the incorporation of data from the physical web onto existing platforms is gradual. We are just beginning to see applications surface and the best is still ahead of us. Below are a few observations, predictions, and implementations of this emerging trend.Sensors Help Keep Elderly SafeThis New York Times article highlights how Seniors are taking advantage of sensors so they can continue to live independently. Sensor systems are able to detect everything from neglected pills to glucose levels to falls. Seniors seem to like the systems, as do their relatives. "In the past, I tried to spend more time on, 'How are you feeling?' " Marvin Joss says. "I still ask those questions, but now it's more to an idea of having a conversation, not trying to listen for clues about whether she's O.K."The Demon-Haunted World
If I had to use one word to describe this presentation by Dopplr's Matt Jones, it would be "Psychogeography," a term developed by French Theorist Guy Debord. Psychogeography is defined informally as "a whole toy box full of playful, inventive strategies for exploring cities...just about anything that takes pedestrians off their predictable paths and jolts them into a new awareness of the urban landscape." Jones cites examples like twittering bridges and pollution sensing robotic dogs to back up a claim by architect Richard Rogers that "Our cities are increasingly linked, and learning." "It seems to me like there are a bunch of hackers reclaiming information from the city," says Jones. "[They are] gardening it without permission."SENSEable City LaboratoryMIT's SENSEable City Laboratory uses sensors to understand the macro-dynamics of cities. For example, in one experiment the lab collected all cellphone usage in Rome for one night. They then aggregated the data and produced a visualization showing how people moved around and where events were taking place. If we had real time access to this kind of information, how would it affect our choices? Would we decide not to eat at a particular restaurant because it is too crowded? Would we choose our entertainment based on the flow of the crowd?AMEE and Google PowerMeter
AMEE and Google PowerMeter are two ways the "here's your data, do something with it" methodology can be used to make people aware of their carbon footprints. Both use sensors such as smart meters to track and display energy consumption over time. (Disclaimer: OATV is an investor in AMEE.)
In a previous partnership between the two companies, Google used AMEE's profiling engine to let users calculate their carbon footprints. After completing the web form, users were taken to a Google Map mashed up with the carbon footprints of those nearby. Soon, we'll be able to do this without the web form. Like O'Reilly said, we are slowly transitioning out of a world where people typing on keyboards are driving collective intelligence.What role do you see sensors playing in your life? How do you interact with them now? Does the possibility of sensor driven collective intelligence frighten or excite you? Post a comment and let us know.
http://feedproxy.google.com/~r/oreilly/radar/atom/~3/t8EOOgq-yUU/radar-roundup-sensors.html
Programming language security, robot laws, open data platform, and telephony recharged:
Languages and Security Reading (Ivan Krstić) -- I love his tripartite division of language security work, as it completely gels with my experience. 1. The “My name is Correctness, king of kings” people say that security problems are merely one manifestation of incorrectness, which is dissonance between what the program is supposed to do and what its implementation actually does. This tends to be the group led by mathematicians, and you can recognize them because their solutions revolve around proofs and the writing and (automatic) verification thereof.
High Time to Act on Armed Robots (New Scientist) -- Philosopher A.C. Grayling (of whom I only know from his appearances on In Our Time) has written an interesting piece calling for us to start talking about the rules and regulations around robots. Not because of any fear they'll enslave mankind, but because we deal with the possibility that people "malfunction" through procedures, expectations, rules, and the law. We don't think much about the failure modes of robots in life, but even less about the legal status of such malfunctions--if an autonomous military robot kills its own soldiers, who is responsible? What are the odds of this happening? This is related to PW Singer's Wired For War. (via Mind Hacks)
Guardian's Open Data Platform -- Everyday we work with datasets from around the world. We have had to check this data and make sure it’s the best we can get, from the most credible sources. But then it lives for the moment of the paper’s publication and afterward disappears into a hard drive, rarely to emerge again before updating a year later. So, together with its companion site, the Data Store - a directory of all the stats we post - we are opening up that data for everyone. Whenever we come across something interesting or relevant or useful, we’ll post it up here and let you know what we’re planning to do with it. They're publishing all this data via Google Spreadsheets, and have a content API to fetch stories. Sample content app built the first day it was public: Guardian + Lucene = Similar Articles + Categorisation I fetched the 13,000 articles categorised as 'Science', fed them to Solr, and used that to generate similar articles and their categories. so if you liked an article you can get another like it. Guardian just put data on universities into their data store. (Via Simon Willison, who worked on it).
Grand Central to Finally Launch as Google Voice (TechCrunch) -- the breathless fawning servile prose of this fellatial article aside, it's wonderful to see telephony apps getting press again (even gush). New features include voicemail transcription, which has to be the new "must have" feature for people like me who live and die (most often die) by the inbox. Voicemail is so due for a reboot, just as much as email.
http://feeds.feedburner.com/~r/oreilly/radar/atom/~3/550253444/four-short-links-12-mar-2009.html
Eye candy, kid brain candy, game code, and some hero worship. If that's a smooth segue to your weekend, you've got a heck of life!
GE's awesome Flash/Computer Vision hack. Has to be seen to be believed (I've embedded the demo video below). (via, via Roger Magoulas)
Eduwear -- "The objective is to develop an educational low-cost construction kit for wearable and tangible interfaces."
GameJS -- A 2D flexible game programming environment in JavaScript, based on Microsoft's XNA Game Network. (via Tom Carden's delicious stream)
Brewster Kahle in The Economist -- he's a saint, behind The Internet Archive, WAIS (the first full-text index engine on the web), and The Internet Bookmobile which was doing print-on-demand before it was mainstream. Brewster FTW!
http://feeds.feedburner.com/~r/oreilly/radar/atom/~3/549323001/four-short-links-6-mar-2009.html
A lot of what I've been working on the past two years has been built on the assumption that the model that social networks use today will fundamentally change. Social networks have largely been built on the premise of being walled gardens in such a way that users can't communicate or share content or friends across networks; put simply this is what keeps a Facebook user from being able to send a message to a MySpace user. This is the same model that destroyed AOL, CompuServe and Prodigy's ISP businesses when normal people chose the Internet itself versus their thoughtfully curated walled gardens.Over the past year we've seen an uptick in the infrastructure, development tools and projects designed to build the social web (n.b. I define the social web as something that is inherently decentralized, just like the web itself). On top of that, MySpace has gone from being off of most developer's radars to the most open social network in existence. With MySpace I'm able to use my account to sign into other sites via OpenID, share my activity using Activity Streams, build applications using OpenSocial, interact with their APIs using OAuth and access APIs that not only allow the creation of new content within MySpace's garden but also extract data from it.While Facebook has made significant contributions to open source projects, ranging from some of their own to memcached, they've largely been absent from much of this progress around building the social web (remember, I define it as being inherently decentralized). Instead, like Microsoft they have willfully ignored many industry efforts in favor of their own proprietary development platforms. To their credit, they've been one of the most innovative social networks over the past two years, pushing the boundaries of what's been thought of as possible with features like social tagging in photos, Newsfeed, Platform, Beacon, integrated chat and Connect.Two weeks ago this changed. Facebook joined the board of the OpenID Foundation, released two-way APIs around status, notes, pictures and videos, hosted a user experience summit focused on OpenID and released a blog commenting widget powered by Connect. Since then they've also talked about how they wish to support the Activity Streams project and have reiterated their commitment to the sort openness that we've been promoting as key pieces of the social web.I know what you're thinking: "talk is cheap." True, Digg said they'd support OpenID three years ago and we've seen...or wait, no we haven't! I wish I had something concrete to point at to show that my next argument isn't crazy, but I don't. All that I can point to is the change I'm seeing when interacting with Facebook and their interactions with developers this year compared to the past. My prediction is that by the end of the year Facebook will become the most open social network on the social web. I believe that not only have they now found business value in doing so, but also truly believe that the next phase of their mission, "to give people the power to share and make the world more open and connected" requires that they do so. This means that anyone building a business based on the notion that Facebook will remain a walled garden and won't adapt - as was true with traditional media when blogging came about - will have their world turned upside down this year.Disagree if you like, but my second argument is that if Facebook does not seriously embrace these ideas this year that their current position of dominance will be usurped. I'm not saying that Facebook will go away, that all of my friends will leave, that it will become irrelevant or that tens of thousands of developers will move on overnight. This year, there is an amazing opportunity to find and define a proper balance between traditional walled-garden social networks and completely decentralized efforts like the DiSo Project.
http://feeds.feedburner.com/~r/oreilly/radar/atom/~3/548906980/facebook-in-2010-no-longer-a-walled-garden.html
Hey, I'm happy to see this in the news today:Amazon.com will begin selling e-books for reading on Apple’s popular iPhone and iPod Touch.Starting Wednesday, owners of these Apple devices can download a free application, Kindle for iPhone and iPod Touch, from Apple’s App Store. The software will give them full access to the 240,000 e-books for sale on Amazon.com, which include a majority of best sellers.I complained about "The Kindle Hardware Tax" earlier. Glad to get a tax cut. Thanks, Amazon.Now, about that DRM......
http://feeds.feedburner.com/~r/oreilly/radar/atom/~3/548776215/kindle-above-the-level-of-a-si.html
Update Sources