Open Sensing City Platform

The Open Sensing City (OSC) platform hosted by Antidot consists in various components. The purpose of the platform is to offer convenient access to Open Data collected, enriched and provided by the OSC project.

OSC AIF workflow for indexing data

The Antidot Information Factory (AIF) is a set of tools whose purpose is to build search engine Index used by Antidot Finder Suite (AFS). The principle of AIF is to design a processing flow of documents (a PDF, a web page, etc...) which will be enriched with various layers. Those layers add informations to the original document in order to use it in the search engine. The enriched version of the document will be used for indexing and creating facets in order to get them back when querying documents through the Antidot search engine. The index is the database used by the search engine to retrieve documents corresponding to a user query.

The workflow is named AFS PaF a.k.a. AFS Pipe and Filters, because documents flows through various pipes which connect various filters. An autonomous set of PaF defines a search feed which is a way to cluster a set of documents in a searchable set. One can define several feeds and query them with the search interface. We can query several feeds all at once or a selected subset of the available feeds.

For OSC project we designed severals feeds depending on the type of [open] data we want to index:

  1. OSC_SITE which contains the indexed data of the OSC Web Site: http://opensensingcity.emse.fr, in this feed a document is a web page with is title, uri and associated text.
  2. OSC_HEAD which contains the [indexed] top-level descriptions of the [open] data sources which are made available through the OSC platform. Each data source is described with a JSON structure describing it:
{"ville" : "Lyon",
"format_fichier_rdf_genere" : "text/turtle",
"format_source" : "application/json",
"type_source" : "temps reel",
"url_fichier_rdf_genere" : "http://opensensingcity.emse.fr/tuba/data/series-rdf/lyon.velo3.ttl",
"date_fichier_rdf_genere" : "2017-03-17",
"ontology" : "http://opensensingcity.emse.fr/ontologies/bikesharing.ttl",
"query" : "http://opensensingcity.emse.fr/tuba/query/lyon.velo3.rqg",
"name" : "Lyon - Disponibilite des velos 3",
"url_source" : "https://download.data.grandlyon.com/wfs/rdata?SERVICE=WFS&VERSION=2.0.0&outputformat=GEOJSON&maxfeatures=1000&request=GetFeature&typename=jcd_jcdecaux.jcdvelov&SRSNAME=urn:ogc:def:crs:EPSG::4171",
"type" : "velo",
"description" : "Disponibilités des vélos à la station JCDecaux 3 à Lyon."}
  1. OSC_VELO, OSC_POLLUTION, OSC_PARKING, etc... which contains data type specific feed found behind the top-level data sources.

General documentation for using Antidot products is publicly available:

The OSC workflow is using up-to-date AFS v9 tools so one should select the appropriate version of the documentation.

OSC AIF workflow design

OSC data type are heterogeneous, i.e. data concerning POLLUTION is not to be compared with car PARKING or even VELO station. That why the OSC_HEAD feed contains an high-level description of the data sources whereas specific type feeds like OSC_VELO or OSC_PARKING do contains specific data.

As usual each AIF/AFS feed contains documents which are layered, this mean that for each document in a feed there may be several layers. Those layers may be retrieved by using the content repository service.

The layers available in each feed is summarized here-after:

The different layers of a document are accessible through the content repository. #### Examples of content repository request

Some feed may deploy a content repository. The content repository enables the retrieval of the various layer(s) associated with a document during the PaF processing. The content repository may be queried using the content Web Service.

  1. Get layer XML_2 for feed OSC_HEAD on document whose afs:uri has been retrieved in a previous search request: http://eval02.partners.antidot.net/content?afs:service=2&afs:status=stable&afs:feed=OSC_HEAD&afs:layer=XML_2&afs:uri=urn:osc:http://opensensingcity.emse.fr/tuba/data/strasbourg/strasbourg.velo.localisation.json

  2. Get layer RDF_1 for feed OSC_HEAD on document whose afs:uri has been retrieved in a previous search request: http://eval02.partners.antidot.net/content?afs:service=2&afs:status=stable&afs:feed=OSC_HEAD&afs:layer=RDF_1&afs:uri=urn:osc:http://opensensingcity.emse.fr/tuba/data/strasbourg/strasbourg.velo.localisation.json

OSC Search engine

The purpose of the Open Sensing City search engine provided by Antidot is to:

There is currently two ways for using the Antidot OSC search engine:

OSC search engine web browser interface

Antidot host a basic set of web pages for testing and visualizing the OSC search engine results. They may be reached at: http://eval02.partners.antidot.net/default/osc/headnew

On this page you have a classical search box where you can type the search query. The default behavior is to search for all OSC feeds. Then you can refine the search by clicking on the left hand-side feeds & facets selection. Since each feed may have its own specific facet, you first have to select a feed which will trigger the display of the appropriate facet set. You can then further filter the results with the facets filters.

The Web Browser interface is built on top of the Web Service interface which is called behind the scene. If one wants to inspect the Web Service called issued by the Web Browser one can open the Network debug tool of the web browser and seek for XHR

OSC search engine web-service interface

The primary mean to use Antidot search engine is to use the Web Service API. Using the web service API you will have access to the whole power of search capability of AFS.

OSC platform offers access to two (out of the four) Antidot web services: - search service - content (repository) service

The search OSC web service base URL is: http://eval02.partners.antidot.net/search?afs:service=2

The AFS tools identify a service with a top-level service Id which is an integer number uniquely identifying the service. This makes it possible to host multiple services on the same host.

Basic AFS search query

The parameters that can be used to call the AFS search Web Service is described here, however you'll find hereafter some sample requests:

  1. Request the documents matching "Rennes" in all feeds: http://eval02.partners.antidot.net/search?afs:service=2&afs:query="Rennes"

  2. Request the documents matching "Rennes" in feeds OSC_VELO: http://eval02.partners.antidot.net/search?afs:service=2&afs:feed=OSC_VELO&afs:query="Rennes"

  3. Request the documents matching either "Rennes" or "Lyon" in feed OSC_VELO using the full text search filter: http://eval02.partners.antidot.net/search?afs:service=2&afs:feed=OSC_VELO&afs:filter=fts(%22Rennes%22)%20or%20fts(%22Lyon%22)

The search web service is answering in XML format as a default but you can request JSON output if you like using afs:ouput parameter:

  1. Request the documents matching "Rennes" in feeds OSC_VELO and get JSON output: http://eval02.partners.antidot.net/search?afs:service=2&afs:feed=OSC_VELO&afs:query="Rennes"&afs:output=JSON

Advanced AFS search query

The AFS search query follows a high level model. One can craft relatively complicated query in order to filter appropriate result in the search answer.

  1. Get documents from feed OSC_HEAD whose facet ville value is equal to "Paris": http://eval02.partners.antidot.net/search?afs:service=2&afs:feed=OSC_HEAD&afs:filter=ville="Paris"

  2. Get documents from feed OSC_VELO for ville equal "Rennes" for which nb parking stand are between 10 and 20: http://eval02.partners.antidot.net/search?afs:service=2&afs:feed=OSC_VELO&afs:filter=nbplace%3E10&afs:filter=nbplace%3C20&afs:filter=ville=%22Rennes%22