Caching the Resource JSON-LD


While in beta there emerged an issue with the WordPress plugin. When installed, the plugin uses client side Javascript to check for Structured data on the server. When it arrives at the Schema App servers, it first goes through a PHP service that does some light touch before doing a lookup in the database, doing some OWL to mapping and then returning to PHP to convert to the familiar JSON-LD compact datasets. The problem with that setup its is fairly resource intensive on the main application server. To offload some of the work, I wanted to use a cache in the PHP web service, and after a couple of hours of work, this is how it came together.

Web Services Middleware Changes

The PHP based middleware was created to allow for logging, security checks, and now a cache. The first two steps were done in PHP, and the last 3 were done on the application server.

  1. In the Schema.Org PHP Controller, add a Cache Lookup and pulls the answer from Cache or Fetches the RDF for that resource from the database.
  2. In the Schema.Org PHP Controller, create a method that will update the Cache given URL and RDF in Turtle format

Application Server Changes

  1. Create a ResourceEditHandler, which handles a request from front end and sets up a SPARQLMotion Script to perform the work
  2. Create a SPARQL Motion Script (SMS) that takes in a resource and graph, fetches the RDF for a resource, then uses the data to call PHP Update Cache Method. This SMS is shown in the image at the end of the post.
  3. In the Editor, subscribe to the event hub, gadgets.Hub.subscribe(“org.topbraid.swa.change” to trigger the ResourceEditHandler, to call the SMS, which calls the PHP Update Cache Web Service

Now, if a URL is called for the first time from the plugin it will lookup in the application server and then save the result to the cache. The cache for the URL is not updated until a resource is created or edited. This will significantly reduce the burden on the Application server for deployments on large websites.


Update January 20, 2016: Some recent reports of slow response times meant another pass at optimization. We did some work in December which included more data in the response from the server that needed conversion. However, the update slowed the routine more than anticipated. In this review, I moved conversion processing out of the cache lookup and into a preprocessor (on updating cache).

The results exceeded my expectations, the test cases showed the speed of response cut by 65% (from 0.350 seconds to 0.125 seconds). In the most complicated case, when many entities cross reference each other, we saw drops from an unacceptable 2-3 seconds to same 0.125 seconds, 20x faster.

, ,
Next Post
Beta Update September 24