At Yahoo!, I spend a fair amount of time on the Web. Not just wasting time on My Yahoo! and Digg, but trying to make sense of the manifold ways that sites expose structured (and not-so-structured) data. There isn't enough agreement about how a web page should describe, say, sports scores or movie showtimes or
WebPath is a little project of mine that provides a way to query the web of structured data hidden in messy markup, from the most simple query (like "does this page use microformats?") up to complex, web-spanning requests (like "can I get to a movie review within two clicks from here?").
The ideas behind WebPath were published at the XML 2007 conference, where lots of great conversations started. Now, more discussions are happening, since the full source code behind the project has been released under a BSD-style license. So, what makes WebPath worth looking at?
Regardless of what type of markup a site uses under the hood, my team -- the Structured Web Group -- looks for ways to pull that structured information into an index where it can be sliced and diced in ways that help searchers get instant gratification via immediate answers to their queries. To accomplish this, the team needed to get a feel for which approaches work best with particular sites. It turns out that as these requirements surfaced, a local Hack Day was coming up, providing the opportunity we needed to try some experimental approaches. In little more than 24 hours, the project progressed from scratches in a notebook to a working implementation.
WebPath at this point is a technical endeavor, suitable for those who enjoy command-line programming. It's written in Python. (It pretty much had to be, given the timeline of its development!) You could consider WebPath a kind of XPath engine, to use the W3C jargon, but its main focus is on the "Web" part of its name. Webpath includes access to a module that accepts messy and invalid pages--just like the ones all over the Web--and "tidies" them up into something easier to work with and analyze.
What's next for WebPath as an open source project? Already developers are getting involved, using WebPath in their own projects, pitching in to offer new features, and, of course, helping fix bugs! I maintain a plain-text todo
list of areas for immediate attention, but in the bigger picture I'd like to wire this up to Hadoop as a front end for truly web-scale queries. You too can help shape the future of this project. If you've taken a look at WebPath, I'd love to hear from you.
-m
Golden Web photo by Cyron.
Popularity: 39% [?]
Built for the most recent Yahoo! Hack Day, NewsGlobe is a fun new way to browse Yahoo! News Top Stories. It pulls together two existing Yahoo! services and takes advantage of the performance enhancements in the latest Adobe Flash Player.
The NewsGlobe consists of three basic pieces: a Yahoo! News Top Stories RSS feed, a geo-encoding web service from Yahoo! Maps, and a free, open-source library of 3D classes for ActionScript 3 called Papervision3D. The application loads the Yahoo! News RSS feed every few minutes and extracts the dateline for each story. It sends this descriptive textual information to the Maps service to find a matching location and thereby return a latitude/longitude coordinate. Then it's simply a matter of using the 3D classes in ActionScript to create a visually engaging experience that's either automated or interactive.
Papervision3D makes it incredibly easy to create a 3D scene, add 3D objects to it, and specify where the camera (i.e., the user's viewpoint) should be located. For each story location where we could discern a lat-long coordinate, we draw a marker object and place it in the proper position on a sphere representing the Earth. The display is calculated and drawn in real time. This allows us to animate the view over time and even let the user change the view by interacting with the objects in the scene.
Since the final product itself is a SWF file, NewsGlobe works online as a web application or off -network as a scaled-down, embedded customizable badge. It could easily be integrated into a Yahoo! Widget or packaged as an Adobe AIR application to run locally on the desktop. By passing in different RSS feeds or search terms, it'd be possible to filter the news and watch stories occurring in a specific part of the world, from a particular category, or matching other keywords.
Who built it
Popularity: 38% [?]
As an engineer for Yahoo!'s Media Innovation Group, I'm lucky to get to use the latest tools available to develop engaging and interactive ActionScript applications. So, I'm surprised that a lot of people still think Flash is best used for screensavers and banner ads. For last month's Yahoo! Hack Day, I decided to show off some of the impressive capabilities of the latest Adobe Flash Player and ActionScript 3 by building a visually interesting new way to browse Yahoo! News top stories. In a surprisingly short amount of time, I was able to mash up two existing Yahoo! services, and then represent the information in a virtual environment I call the NewsGlobe.
The NewsGlobe consists of three basic pieces: a Yahoo! News Top Stories RSS feed, a geo-encoding web service from Yahoo! Maps, and a free, open-source library of 3D classes for ActionScript 3 called Papervision3D. The application loads the Y! News RSS feed every few minutes and extracts the dateline for each story. It sends this descriptive textual information off to the Yahoo! Maps service to find a matching location, and return latitude and longitude coordinates for it. The rest is simply a matter of using the 3D classes in ActionScript to create a visually engaging experience that's either automated or interactive.
Papervision3D makes it incredibly easy to create a 3D scene, add 3D objects to it, and specify where the camera (i.e., the user's viewpoint) should be located. For each story location where we are able to discern a lat-long coordinate, we draw a marker object and place it in the proper position on a sphere representing the Earth. The display is calculated and drawn in real time. This allows us to animate the view over time and even lets the user change the view by interacting with the objects in the scene.
Since the final product itself is a SWF (Shockwave Flash) file, NewsGlobe can live online as a web application or be embedded off-network as a scaled-down customizable badge. It could also be integrated into a Yahoo! Widget or packaged as an Adobe AIR application to run locally on the desktop. By passing in different RSS feeds or search terms, it could be possible to watch stories occurring in a specific part of the world, from a particular category, or matching other keywords.
In all, NewsGlobe was a fun project to test a simple idea and see if I could achieve the desired result within a short amount of time with relatively little difficulty. Is it perfect? No. Could we add a lot of additional features to make it something more useful and accurate? Absolutely. However, I would certainly consider this a successful proof of concept and viable prototype for similar future interactive 3D visualizations.
If you want to learn more about the individual pieces used to create the NewsGlobe, follow the links below:
Popularity: 40% [?]
Lucas J. Shuman, February 12th, 2008 on 6:23 pm
What's next? This isn't a rhetorical question: after today's release of the Yahoo! ActionScript 3 Maps API , it's a question we feel we can ask. With a native ActionScript 3 mapping engine, map control widgets, geocoding, local search APIs, and custom markers and overlays, what can possibly be next? And the answer is simple: whatever you build with it!
The new component is very lightweight (base size is 30Kb), yet incredibly functional, easy to use and opens up a number of the other geo-centric web-services we have at Yahoo!, making it easy to develop overlays and components that build on top of the API in any of your Flash, Flex, AIR or Ajax applications.
To give you an idea of its functionality, during the development of the new API, Jonathan New and Benjamin Halsted of the Yahoo! Messenger team hacked up a really cool flick-able and rotatable map interface for our internal Q2/07 Hack Day, and won for the most fun hack! You can check out their flickable maps hack right here. Try to toss the map around, and shift-drag to rotate around the center. Special thanks go out to them for making such an awesome app.
If you have some Flex or ActionScript development chops, definitely jump in on the fray and grab a copy of the AS3 Maps API for yourself. We've prepared a few screencasts, examples and complete documentation to get you started, and we look forward to feedback.
I would also like to thank a number of people that made this release possible. First, everyone on the Yahoo! Maps and Flash Platform teams, past and present, including Mark Law, Rodney Fernandez and Allen Rabinovich. Huge thanks also go out to Lawrence Morrisroe, Buck DeFore, Ted Patrick, Eddie Babcock, Randy Troppmann, Chuck Freedman, Aaron King, and JR Conlin. Whew.
Popularity: 34% [?]
Zach Graves, February 11th, 2008 on 6:00 am
Yossi Vardi - prime mover at ICQ many web years ago (recently covered by Scoble at Davos) quoted Teddy Roosevelt at a Techcrunch event a few months back:
"It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat."
Nice going, hackers. You guys are -- seriously -- the reason why I work here at Yahoo!.
(Slideshow photos from Maximum Mitch )
Popularity: 13% [?]
Kent Brewster, January 28th, 2008 on 12:05 pm
Who We Are
Search
Stay Connected
Recent Posts
Next* Links
Our Community
When / Where
About Next*
