Mashups

Derek Bridge

Department of Computer Science,
University College Cork

Mashups

Aims:

• to learn about mashups, web services, APIs and screen scraping
• to study an example of a mapping mashup

Mashups

• A mashup mixes resources from different sources to produce a new resource
• Music mashup (similarly videos)
  • mixes existing music to produce a new piece of music
• News feed mashup (RSS/Atom)
  • mixes data from multiple feeds into a single feed
• Webapp mashup
  • mixes data and services from multiple sources to produce a new webapp

Mashups

• The most common webapp mashups are mapping mashups, based on Google Maps, Microsoft Virtual Earth, Yahoo! Maps, etc., e.g.
  • www.housingmaps.com
  • www.football-maps.co.uk
• But others are based on photos, videos, shopping, etc.
• E.g. http://mashupawards.com/

Building a mashup

To build a mashup:

• Find a site or sites that host data or services that you want to reuse
• Write HTML, CSS, server-side scripts (e.g. in PHP) and client-side scripts (e.g. in JavaScript) that you host on your server
• Your scripts will get data from the other sites or access services (i.e. run programs) on the other sites
• If you're lucky, the other sites will have an API
• If you're unlucky, you will have to resort to screen scraping

APIs (Application Programming Interface)

• An API specifies how a programmer can use a library of other programs, functions, classes, methods, etc. when writing his/her own program, without knowing the details of how the resources in the library are actually implemented, e.g.:
  • The Java API: http://java.sun.com/javase/6/docs/api/
  • The Google Maps API: http://code.google.com/apis/maps/documentation/reference.html
• Innovative sites (e.g. Google, Flickr, Amazon, eBay, YouTube, Microsoft, Yahoo!) offer an API to faciliate the development of mashups

Screen scraping

• If there's no API but you want data from someone else's web site, what do you do?
• You write a screen scraping program, which
  • fetches web pages (designed for human consumption) from the other site
  • parses the web pages (i.e. breaks them up into the constituent parts) to extract the data you need
• Class exercise: This is much less satisfactory than having an API. Why?

My database-driven mapping mashup

CREATE TABLE markers (
   id INT NOT NULL AUTO_INCREMENT,
   description VARCHAR(100) NOT NULL,
   latitude DOUBLE NOT NULL,
   longitude DOUBLE NOT NULL,
   PRIMARY KEY (id)
);

fetch_markers.php, a PHP script that returns the contents of the database

<?php 
 // Include the PHP to connect to the database
 include('dbconnect.php');
 // Retrieve all markers
 $sql = "SELECT * FROM markers;";
 $dbresult = mysql_query($sql);
 if ( ! $dbresult )
 {
    die('');
 }
 // Output each database row, with commas and vertical bars as separators
 while ($row = mysql_fetch_assoc($dbresult)) 
 {
    echo "{$row['description']},{$row['latitude']},{$row['longitude']}|";
 } 
?>

store_marker.php, a PHP script that stores data in the database

<?php 
 function clean($str)
 {
    return mysql_real_escape_string(trim($str));
 }
 
 // Include the PHP to connect to the database
 include('dbconnect.php');
 // Validate the data again, even though it was done on the client (belt and braces)
 $desc = clean($_GET['desc']);
 $lat = clean($_GET['lat']);
 $lng = clean($_GET['lng']);
 if ( $desc == '' || 
      ! is_numeric($lat) || $lat < -90 || $lat > 90 ||
      ! is_numeric($lng) || $lng < -180 || $lng > 180 )
 {
      die('');
 }
 // Try to insert into the database
 $sql = "INSERT INTO markers (description, latitude, longitude) 
         VALUES ('{$desc}', {$lat}, {$lng});";
 $dbresult = mysql_query($sql);
 if ( ! $dbresult )
 {
      die('');
 }
 // All is well with this new marker
 // Include the cleaned description in the response
 echo $desc;
?>

db_driven_markers.html, the web page that displays the map

This page links to two scripts written in JavaScript:

• One is supplied by Google, for which I have to use my Google Maps API key (obtained from http://code.google.com/apis/maps/signup.html)
• The other is written by me: it's the script that brings everything together!

...
<head>
 <title>Database-driven markers</title>
 <link rel="stylesheet" type="text/css" href="db_driven_markers.css" />
 <script src="http://maps.google.com/maps?file=api&v=2&key=ABQIAAAAPuAB5dIIlSZlic4C-yzmcRSY3ZrWNGZjOumIhKGxoGSDJVT-4RRYo5lY6lmmd_gaJoVRlw1NQUc_PQ" type="text/javascript">
 </script>
 <script src="db_driven_markers.js" type="text/javascript">
 </script>
</head>
...the rest is just an HTML form...

db_driven_markers.js, client-side JavaScript that brings it all together

• When the document loads, the script
  • creates the map
  • uses an XMLHttpRequest to run fetch_markers.php to get all the markers from the database
  • puts these markers on the map
• When the user enters data into the form and presses the Add Marker button, the script
  • validates the user's data
  • uses an XMLHttpRequest to run store_marker.php to store info about the new marker in the database
  • puts this new marker onto the map
• When the user clicks on the map,
  • if there's already a marker at that position on the map, the script puts info about the marker into a balloon and into the form
  • if there's no marker at that position, the script puts the co-ordinates into the form
• (A more professional version would use a GMarkerManager object to more efficiently handle the set of markers)

Web services: the future?

• The web is a platform that gives access not just to information, but also to services
• Increasingly, the web is used for app to app communication
• Web services are self-contained apps that perform useful functions (e.g. small ones like currency conversion, and large ones like language translation)
• They run on some host server
• The services are registered in directories on the web so that others can discover them (see UDDI: Universal Description, Discovery and Integration Service)
• A description of the service (e.g. its API) is returned to the client (see WSDL: Web Services Definition Language)
• The client invokes the service across the web using some protocol (see SOAP, which once stood for Simple Object Access Protocol, or, more commonly, REST, Representational State Transfer)
• Most of the markup languages used are XML markup languages

Web services: the future?