How to Automate Schema Markup at Scale and with Speed

Do you have a lot of content on your website and want to optimize it with Schema Markup but just can’t get it done? Maybe you don’t have a schema ninja on your team or are restricted IT/developer resources in your company or your customers. Well, you are in luck! There are approaches for doing schema markup across hundreds of thousands of pages with or without IT and using Tools that act as your expert!

The Schema App Team has tried all of these approaches and no one size fits all.  Our goal is to help you understand the options available, their benefits and tradeoffs, so you can make a business decision with the knowledge that you are picking the right solution for your content and your business.  Let’s take a look.

 3 Things to Consider when Automating Schema Markup

#1 MAINTENANCE

In order to maintain the health of the schema markup, you want to ensure it is updated when the page content changes. Each approach has a different level of effort to maintain schema markup. Choose which solutions fits your unique situation.

#2 IT DEPENDENCIES

It is unlikely that you have endless hours of IT resources to implement Schema Markup. Consider approaches that allow your marketing team to adopt schema markup with little to no IT resources.

#3 SPEED TO RESULTS

There are some automation approaches that still require the engagement of technical resources, approvals for data and more. Speed to market and results depend on the approach.

Automation Solution Comparison

To help you consider the best approach for your business, we’ve put together this chart, that compares the different approaches, IT/dev effort required and the amount of time to implement the solution.

Approach
Client Effort
Maintenance Option
Search Engine Visibility*
IT Involvement
Microdata in Template High Manual ALL High
Bulk Upload High Manual ALL Medium
Data Source Low Automated ALL Medium
Data Feed Medium Automated ALL Medium
Crawl Low Automated ALL Low
Google Highlighter Low Manual Google Low
Schema App Highlighter Low Manual ALL Low

*Today, Google, Bing, Yahoo and Yandex all have different support for schema markup formats (JSON-LD,Microdata,RDFa) and Synchronous vs Asynchronous loading of data. As such, while your schema markup may be on a webpage, it may not be “seen” by the search engines because of their level of support.

Schema Markup Automation Solutions

Plugins and Apps

Using plugins and Apps for automating Schema Markup is a great way to automate schema markup. However, these options are only available on common e-commerce and CMS platforms such as WordPress, WooCommerce, Shopify, etc. Ensuring that the plugin or Apps produces accurate JSON-LD, maintains the plugin and makes the markup available to all search engines is key. In order to determine if a plugin or app is the right approach, determine what schema markup you want on your pages before you get started, then compare your desired markup with that in the plugin to see if it provides detailed enough markup. 

Schema App offers a WordPress Plugin, WooCommerce Plugin and Shopify App.

To find a list of commonly used plugins, have a look at the 70+ list of tools for schema markup.

Pros & Cons

Pros:  Plugins and Apps are great because they usually are easy to setup and use the data already in your webpages.

Cons:  Plugins and Apps often meet the basic requirements for schema markup but don’t allow customization. Depending on the types of pages you have on your website and how specific you want to be in your schema markup, a  plugin or App may not have detailed enough markup available to meet your desired outcomes.

Hardcode Microdata in HTML Page Template

Microdata is a set of tags that you can add to your html to add schema markup to your pages.  Microdata is what Google originally recommended in their Structured Data Documentation. More recently, Google is highlighting JSON-LD as their preferred Schema Markup syntax. The problem with microdata is that you need to know how to code, need to have access to the website and be able to maintain the code over time. We use this process for smaller sites, where there is only one person making changes to the site and are comfortable coding. 

Pros & Cons

Pros:  Data is always up to date, easy way to provide schema data. The original Schema at Scale option

Cons:  Requires a technical IT or developer resource to implement.  As a result of the microdata being inline with the HTML, it is hard to maintain and can be impacted by page layout changes.  Web Designers seldom consider the impact of page changes so change management or different actors become an issue. Microdata is also no longer the preferred syntax from Google.

Bulk Uploads

You can use a bulk upload of your data to produce JSON-LD. Schema App’s bulk upload allows you to create a capture of the data for your page data item (location, product, articles, etc) in csv format, where each column is a property for the page type.  The resulting schema markup is in JSON-LD and can be deployed through Google Tag Manager, WordPress Plugins, Shopify Apps, or a Javascript API call.  New Data can be uploaded at any time to keep schema markup up to date. It has the ability to automate the updates, by checking a remote directory on a schedule. User has the option for full updates or changes (deltas) only.  Bulk Upload can be used for any schema markup class and all schema.org extensions (Health, Automotive, etc).

Real Life Example

A website has a lot of content that you want to optimize with schema markup. A bulk upload can be created for article markup from a download from the website database. Since this information doesn’t change often, updates can be made manually in Schema App without a lot of effort.

A company has 162 locations and wants to add schema markup to each of the sites. There is no access to the page source, and the client is very sensitive to cost. The client can fill in the excel spreadsheet with information about their locations and agency can make manual updates if required after upload.  

Pros & Cons

Pros: No developer/IT resources required. Simple process and layout – can be done by a junior resource. Easy to make small updates once it is in Schema App.  Allows you to use the entire schema markup vocabulary and properties to fully describe your web content.

Cons:  Need to do manual updates or a new upload when data changes.  Hard to maintain if data changes frequently.

Converting a Bulk Data Set to Schema Markup can take different forms. The input data set is typically a group of data resembling the same entity type, e.g. Recipe or Product or LocalBusiness. You might have a Spreadsheet or TSV / CSV lists of data that needs to be converted.  

The conversion process as we do it, goes through a few steps:

  1. Load the file into the transformation script
  2. Convert the tab separated list to an RDF Graph
    1. Provide Data Set’s primary Schema Class
    2. Each row in the spreadsheet will be converted into an instance of the class, the subject
    3. Each column represents a property, the predicate
    4. Each cell is a value, the object
  3. Perform some data checking and cleaning, e.g. assign the @id
  4. Map the input graph to schema properties
    1. Check for properties that end in a number, e.g. sameAs1, sameAs2, to convert to schema properties
    2. Check for properties that are known schema properties. e.g. name, url, telephone
    3. Look for expected mappings of more complex fields, e.g. AggregateRating, PostalAddress, AdditionalProperty, and construct subdata items with their respective properties
  5. Update the RDF Database for the account
  6. Update the Schema App Cache, by exporting the RDF data to JSON-LD for each data item
  7. Store information about the Import Process Job

Everything except for 4c are common for all data import jobs, regardless of the class and properties being imported.

The Schema App process allows you to create a bulk template for any schema markup class and any combination of properties. It provides you the flexiblity and the opportunity to use the data on your website optimally with schema markup. The spreadsheet below is an example of the data you can import.

In order to maintain the data, you can upload the data in its entirety or those URLs that have changes. Alternatively you could login to Schema App and make manual changes.

Learn more about Schema App Bulk Upload Capabilities

Automate Schema Markup at Scale!

Website Crawl

Schema App crawls the website and discovers data off the pages. No direct access to database is required. You can use one of many tools to crawl a website and extract data, such as Import.io, ParseHub or build your own using a Scrapy.org type framework. They will help with setting up a page template based crawler, to collect a list of Recipes, BlogPosts, Products, etc. These templates can produce a Data Set that can be converted to the schema.org model and schema markup to be placed on your website. Schema App takes the crawl data and transforms it to JSON-LD using the aforementioned Bulk Upload Process. The resulting schema markup is in JSON-LD and can be deployed through Google Tag Manager, WordPress Plugins, Shopify Apps, or a Javascript API call.

Real Life Example

A customer has a large amount of health data on their website. Agency does not have IT resources on a customer site to add and maintain micro data. They want to add JSON-LD at scale across the site.  By identifying the page type, and page URLs, Schema App can extract the data from the page with no IT involvement and maintain the markup through subsequent crawls. 

Pros & Cons

Pros: No Developer/IT resource required.   

Cons: Site can block crawls on a page.  Crawl needs to be done regularly to keep data up to date.

Google Data Highlighter

Google Data Highlighter provides users the ability to highlight the fields on their webpage and map it to schema markup. The resulting markup is in JSON-LD and is only made available to Google Search Engines (not Yahoo, Bing and personal assistants). The schema markup vocabulary available in the data highlighter is very basic, and covers only a handful of the structured data features offered by Google.

Pros & Cons

Pros: No IT/Developer required. Easy to use.

Cons:  The resulting markup is in JSON-LD and is only made available to Google Search Engines (not Yahoo, Bing and personal assistants). The schema markup vocabulary available in the data highlighter is very basic, and covers only a handful of the structured data features offered by Google.

Schema App Highlighter

Similar to Google’s Data Highlighter, the Schema App Highlighter allows users to highlight any field on the webpage and map it to any schema markup property. Highlighter produces JSON-LD and can be deployed through WordPress plugin, Google Tag Manager, Shopify App, or Javascript API call.  

Real Life Example

Digital Agency is working with a large enterprise and wants to prove the value of investing in Schema Markup. The Agency does not have access to the website back-end and the site is large and in the medical field.  They decide they are going to optimize the Medical articles on the site with schema markup. They use the Schema App Data Highlighter to map schema markup to the article structure and then upload the list of 13,000 articles that have the same template and layout.  They complete the mapping and deployment in less than 48 hours.  The Agency does  have access to the client’s Tag platform, Tealium, and deploy the Schema Markup through a custom tag.  Finally, they measure the value of schema markup by adding custom dimensions to the Google Analytics deployment to show the difference in performance for pages with schema markup and those without.

Pros & Cons

Pros: No IT/Developer required. Easy to use.  Schema App Highlighter has the entire vocabulary and properties.  Very fast to deploy. 

Cons: JSON-LD created is available to all search engines.

Learn more about Schema App Data Highlighter

Automate Schema Markup at Scale!

Industry Product Feed – Google Adwords, RSS, etc.

If you have a data feed, for example, AdWords Product Feed, RSS/Atom, Schema App can transform it into JSON-LD This process can be created to be triggered anytime new data is added to the data source.  What is great about this approach is that you can get the basic schema markup done quickly, with no IT resources.

Real Life Example

Agency does AdWords for the client and now wants to add Schema Markup to the products.  They have already created the AdWords Product Feed, and now can reuse it to create schema Markup in JSON-LD. No IT resources are required because the client site is already using Google Tag Manager for analytics.  The agency uses Schema App Data Feed integration to create the schema markup in bulk and deploys it to the site through Google Tag Manager.

Pros & Cons

Pros: Data is already packaged and ready to go. No engagement with IT required.

Cons: Data in the feed may not be comprehensive and might have to be augmented with data from other sources.

Data Source, Bulk Convert / Custom Code

If you have a data source with all your page content, you can use Schema App to transform the data feed into JSON-LD. The Data Source, such as a MySQL or Postgres Database can be accessed through a read-only Database View so that a process can be run to create schema data. In some scenarios, this process might be a Bulk Set, similar to the Bulk Upload process defined earlier.  The JSON-LD can then be deployed to the site through Google Tag Manager, WordPress Plugin, Shopify Plugin or Javascript API call.

In other scenarios, you can use the web server’s programming language, e.g. (PHP, Ruby, Python) to custom write code which renders the schema markup during each page load using calls to the Data Source. Many schema plugins and apps used this method to generate JSON-LD dynamically for each page template type.

Real Life Example – Data Source

If a database is available with the page information, then this data can be read through a read-only database view and can be used to create schema markup.

Pros: Allows you to create the schema markup at scale with all the data on the pages.

Cons: Maintenance is hard. Every time there is a change in the data, it needs to be uploaded to generate new schema markup.

Real Life Example – Custom Code

In order to add schema markup to products on a web platform that didn’t allow javascript calls, or apps installed, we added custom code on the page to take the fields on the page and generate the JSON-LD.  

Pros & Cons

Pros:  Allows you to create schema markup amongst the limitations of a platform and do it at scale.

Cons: Maintenance of the code is painful. Changes to the platform might impact the code, or changes to the page layout.

Ready to explore how to do Schema Markup at Scale?

Book at Discovery Call

So that’s how you can automate schema markup! See how the solutions range in complexity and benefits? We love helping companies overcome their challenges doing schema markup at scale. We often get asked which approach is our favourite? While we like most technical challenges, we love the elegance of automating the schema markup off an existing data feed and empowering anyone to do schema markup at scale with the Schema App Data Highlighter.

Did we miss any approaches you use? Let us know at info@schemaapp.com.

Menu