Shopify — state of play — 2018/2019
The diaries of tech-savvy e-commerce developers 🐐
If you’re looking to build an e-commerce shop in 2018, there are a variety of platforms and tools you might end up using. One of the names that will keep popping up — from Kylie Jenner’s in Forbes $1B story to the Tesla Motors shop — is Shopify, described as a one stop shop for all your e-commerce needs.
At Airnauts we’re constantly approached by brands, companies, startups and individuals who wish to start selling online or transform their current setup to something more modern & cool.
Our e-commerce experience dates back to the early 2000’s when one of our co-founders launched one of the first Polish e-commerce platforms.
Since then we’ve come a long way but we have never stopped looking for the best solution to run a great shopping experience online. We’ve worked with Magento, WooCommerce, Spree Commerce, PrestaShop and Shopify and after seeing more and more clients being drawn to that last solution, we’ve decided to share our observations regarding the platform. We mainly work with Shopify Plus, the more advanced, enterprise version of the platform, so we’re looking at the full spectrum of possibilities that the platform has to offer. We also use a variety of the existing Shopify Plugins from the Shopify App Store and, when needed, even build our own.
First off, there are two ways to build a shop with Shopify — either by building a theme directly via their templating language (Liquid), or by developing a custom frontend and integrating it through Shopify APIs. We’ve done many projects with both approaches and would like to share our findings below.
The Shopify Template approach:
The first important distinction is to understand the difference between a template and the actual functionality of a shop. A template, as the name suggests, is merely a styled frontend displayed to end users and the functionality is limited by the platform and elements that can be used in a template. We are used to building complex single page applications in Angular, React and Vue so for us it was easy to forget that we shouldn’t inject advanced Javascript logic into the frontend through the templates. However, sometimes it is the only way to achieve certain functionality if it’s not supported out of the box by Shopify. For an example, we recently worked on a high-end jewellery shop which wanted to sell hundreds of products so filtering & sorting was required. Usually one would expect that such rather basic features would be available — unfortunately, this was not the case. We needed to add a whole bunch of custom JS scripts just to be able to manage such operations on product collections, definitely exceeding the boundaries of what one would consider a template. More specifically, we ended up writing a script which fetches all of the products from a collection and filters them on the client side. For each filter parameter, we iterated over all of the fetched objects. This type of workaround would definitely be considered bad practice and — aside from weeping developers — also has a negative impact on performance of the site. That’s why it is strongly suggested to simplify any sorting and filtering logic as much as possible. If you look at some of the more popular Shopify shops, be it, for example, the aforementioned Kylie Jenner store — you’ll notice you always browse through very flat lists of merchandise, simply grouped into collections and with some very basic sorting available. It’s not a coincidence every other successful shop running on Shopify looks similar in that regard — it’s just a limitation of the platform and trying to hack your way around it is always a bad idea.
Another thing to consider are plugins. When discussing an e-commerce project, a misperception commonly occurs regarding the magic powers of the previously mentioned Shopify App Store — “Surely, there must be a plugin for everything available out there!” The marketplace is a great idea, and we love the concept of allowing third-party developers to enhance the platform. However, we shouldn’t forget that those plugins, although reviewed by Shopify before being published on the store, are developed and maintained by their creators. You have no control whatsoever, so security, stability and performance is in the hands of others. This means that your data is stored somewhere in a database under the care of an unknown developer or company and the plugin itself may stop functioning properly at the worst possible moment. Also, the logic of the plugin needs to exist within the same limitations of Shopify so most of the time what happens under the hood is simply awful, for example, hundreds of tags being automatically added to products, which are later read and used based on some string-comparison logic. And of course, those tags could be manually edited at any moment, which would break everything. This is just one example and there are many others out there.
One example where a plugin is often needed is for the use of metafields — additional custom fields that can be added to products.
They are a necessity because it is impossible to add custom product attributes as separate data. This means all product information needs to be stored in a single field as a large blob of text. This also means that there is no option to compare product attributes (for example, in a store selling electronics, you cannot compare two devices by their technical specifications). Metafields are a very nice feature to store simple additional data for products or other types of objects (like pages, customers, etc.). Unfortunately, to handle metafields, as mentioned above, a plugin is needed as there is no native way to manage them. Liquid templates can handle viewing metafields but to create/edit them you need a third-party app from the Shopify App Store that will allow you to manage metafields stored in your shop.
Although they are very valuable, metafields are only a substitute for actual attributes. They cannot be used for any operations on the inventory and they have their own limitations — for example, they are limited to storing only data of types String and Integer. That’s a frustrating limitation because oftentimes it would be much more practical to use data structures such as arrays or dictionaries. For example if you want to store some grouped data for a given product — e.g. a bulleted list — you will need to store each bullet in a separate metafield. Again, there is a way to hack your way through but it is far from ideal. Values can be stored as strings separated by a special character or string — e.g. value_1###value_2###value_3where ### is a separator. Such a string can be then split in Liquid by a split filter: {{ metafield.key | split “###” }}. Obviously, that’s awful and it could backfire with errors. To add another layer of complexity, Liquid also lacks any JSON support. It’s parsed to plain text, which makes it harder to manage and maintain. This also means that any further form of nesting data is impossible.
Similar to metafields, there is no hierarchy for collections — it’s impossible to have a collection within a collection, which would be extremely useful in many cases. This extremely flat structure causes many problems in terms of the efficiency and general information architecture, which quickly becomes a huge mess, especially with larger data sets.
Handling a larger product catalog is a problem in general as one Shopify product can possess no more than 100 variants. Each variant of a product can be conveniently created by setting up custom attributes such as for example color or size, but you can’t create variants made of more than 3 attributes. In one of our projects we ended up with yet another hacky workaround where we created an additional collection for every product and each product in this supplementary collection represented a variant. In addition, we had to add a tag which described the additional attribute, and glue it all together on the frontend. This solution, although functional, made managing the product catalog a much more complex & error prone effort.
Another issue with a large product catalog is custom filtering. This needs to happen on the frontend and to make it work, basically, the client needs to first fetch all of the products (with{{ all_products }} in the template) and then loop and filter. Performance-wise that is a very poor idea.
While working on the product catalog itself, another thing to remember is the limitations of the WYSiWYG editor. Although it gives some flexibility, it’s very simple and gives only the basic options e.g: text styling to bold, italic etc. There is no way to create hyperlinks, galleries or even lists. Aside from the copy, while setting up product images Shopify gives you a very convenient gallery to work with. However, it’s impossible to upload any HTML tags via this gallery so there isn’t a way to setup super-sharp SVG icons. Also, there is no concept of vendors as separate entities (with their own image, description etc.) for the product catalog built into the platform. So, for example, it’s impossible to easily create a page showcasing all of the vendors and linking out to their products.
Similar to metafields, the live preview for sections has comparable limitations. We love the fact that the Shopify admin panel gives us a live sections editor, which in our opinion is one of the best features of the platform. It’s very well designed and super easy to use. From a development standpoint, while working on a Liquid template you can simply create a schema tag in your section template and add some moveable elements to it, but… there is no way to nest one section inside another and create repetitive fields. The structure of these sections remains flat. Each section is built from chunks which are scoped to their parent. There is no way to create globally-scoped fragments which can be utilized as blocks that you can use freely in every section. This would be an extremely useful and powerful feature. Unfortunately, that cannot be done and you need to copy/paste that type of block into the schema tag of another section.
Another problem we’ve encountered is pagination. It’s built into Liquid and seems like a pretty basic feature, however, to achieve smooth and elegant loading — which, being a company that creates appealing SPA web apps on a daily basis, is what we expect — more hacking is required. If you’re looking for continuous lazy loading on scroll or would like a “load more” button that doesn’t reload the entire page by jumping to a different URL, you will need to get creative. Our workaround is basically a JS script that searches within the HTML DOM of the page for a link to the next page, fetches the entire page and then renders the content. Also, there is a limit on the results per page which cannot be increased. In general, navigation gets problematic if anything custom is needed. Liquid gives you easy access to navigation Objects, which is really convenient, however, problems start to crop up when you want to re-organize elements inside such objects, for example, doing some custom nesting or including additional data (images, custom description, etc.). It just cannot be done.
Assuming your shop has global aspirations, and that should be the general assumption for the vast majority of e-commerce shops out there, ideally you would like to handle various currencies and languages. However, one shop can support only one currency, which means you will need to create and manage a copy of each product in a separate shop just to set another currency for it. That causes immense problems if you use Shopify in your frontend application and want to dynamically change currency. Also, just managing the product catalog becomes a nightmare. Recently Shopify has been working on introducing multi-currency support in their enterprise product, Shopify Plus: https://www.shopify.com/enterprise/international-ecommerce-strategy
With Multi-currency for Shopify Payments, you’ll soon be able to grow globally by acting locally.
Even if this issue is finally resolved soon, we are still left with incomplete multi-language support because, although Shopify has a great built-in filter for translations, it doesn’t handle any dynamic content such as metafields.
One of the most important moments for any e-commerce site is the final checkout experience. Shopify has done great work to make this process streamlined, convenient and easy to implement. However, to apply any decent styling you will need a a Shopify Plus account. Otherwise, you’re stuck with a generic styling which can be different from your general brand aesthetics and the look of your store. Additionally, even if you have the proper account type, the styles for the checkout step are kept in a different file which supports SASS syntax but… it supports the very first version of SASS, which is very different from the modern syntax. Finally, even if you get the styling just as you like, the complete checkout form with inputs (shipping address, etc.) renders itself as an Object and you are unable to modify anything about it. For example, adding custom product data to the form is impossible.
Obviously, these limitations make developers cry worldwide:
As always, Shopify is so annoyingly restrictive. I have to have one of the most important pages on the site the way Shopify says?! At least give some level of control over the content like accordion sections!!!
To summarize, building a Shopify store utilizing Liquid templates can get your shop up and running very quickly. However, it needs to be a very basic shop without any advanced filtering or sorting, handle a single currency with a predefined checkout, and preferably have a small product catalog with a flat hierarchy and few variants. Also, your life will be made much easier if the size of your catalog is small enough to be able to avoid any paging. If this is not what you intend to build, Shopify is probably not for you.
The custom frontend with Shopify API approach:
As mentioned before, at Airnauts we usually build advanced single page applications, often times both the backend (CMS/CRM, API) and the frontend at the same time. That’s why it was obvious to us that, although Shopify Liquid templates come with limitations, for more advanced builds we could certainly create an amazing frontend and hook it up to the Shopify API. Unfortunately, there are numerous caveats and limitations to this approach as well.
The first important piece of information is that, in fact, there are two different APIs — a RESTful Admin API and a Storefront GraphQL API. We believe GraphQL is a game changer and have been internally using it in every project for quite some time so we were definitely very drawn to this approach. However, to build a cohesive store you will need to use both APIs at the same time and each will need to be used for a certain part of the job. Although initially this split has some logic to it (managing content versus displaying content requires different types of access) eventually it causes a lot of issues as these APIs return incomplete and misaligned data. For example each equivalent of the same object in these APIs has a different ID (coded in Base64).
As mentioned in the previous section, Liquid templates are very limited when it comes to customizing the checkout look and user experience. Unfortunately, things do not get better here as the only option you have is to redirect the user to a checkout page and then redirect the user back after the purchase. So, all prior limitations still apply and after factoring in the redirects the user experience is even worse.
The Shopify APIs are very simple. For example, you can’t define and use custom query params in GET requests. Nor can you use any of the metafields for your queries. What is worse, there is no way to get all of the data of an Object in one request. For example, if you fetch product details you will not receive any of the custom data stored in the metafields. For that, you will need to make another request to a separate endpoint. This is all you’ll get for a product:
Product:
{
“product”: {
“id”: 632910392,
“title”: “IPod Nano — 8GB”,
“body_html”: “<p>It’s the small iPod with one very big idea: Video. Now the world’s most popular music player, available in 4GB and 8GB models, lets you enjoy TV shows, movies, video podcasts, and more. The larger, brighter display means amazing picture quality. In six eye-catching colors, iPod nano is stunning all around. And with models starting at just $149, little speaks volumes.</p>”,
“vendor”: “Apple”,
“product_type”: “Cult Products”,
“created_at”: “2018–07–05T12:41:00–04:00”,
“handle”: “ipod-nano”,
“updated_at”: “2018–07–05T12:41:00–04:00”,
“published_at”: “2007–12–31T19:00:00–05:00”,
“template_suffix”: null,
“tags”: “Emotive, Flash Memory, MP3, Music”,… }}
And here are the metafields as a separate response:
{
“metafields”:[
{ “id”: 625663657,
“namespace”: “translation”,
“key”: “title_fr”,
“value”: “tbn”,
“value_type”: “string”,
“description”: “French product image title”,
“owner_id”: 632910392,
“created_at”: “2018–07–05T12:41:00–04:00”,
“updated_at”: “2018–07–05T12:41:00–04:00”,
“owner_resource”: “product_image”,
“admin_graphql_api_id”: “gid://shopify/Metafield/625663657”
}
]
}
Not great.
Another serious limitation of the API is the search functionality. You cannot make a single request with several search params and get a complete list of results. It get’s especially complicated if products are somehow related between each other — Shopify allows you to create connections between products, but to retrieve them from the API, you will need to fire a series of GET requests.
It is also impossible to fetch more than 250 Objects at a time — be they products, collections, variants or metafields. Of course, you can create additional requests to fetch the rest of them but things get tough quickly, for example, when you want to make some filtering available to the end user. In this example, if you build out custom frontend filtering for your products, you’ll obviously want to filter all of the products in your store. But if there are more than 250 products, you first have to fetch all of them, page by page, which will affect the performance of the shop. Even just to get the count of all of the products in your shop or a specific collection, you need to make another request to a separate endpoint. What is more, there is a 40 requests per minute limit, which is problematic due to all the other limitations and the need to often make multiple queries to fetch the data you need. The Shopify SDK can help with that by creating a queue of requests to ensure that you do not go over the limit, however, that doesn’t solve the issue itself.
API call limit
Shopify's API has a call limit that uses a leaky bucket algorithm. The bucket size is 40 calls with a leak rate of 2…
https://shopify.dev/docs/api/usage/rate-limits
As you can see, even with a very edgy frontend build, a great, modern stack and good HTML, CSS and JS, when it comes to the shopping experience itself the limitations of the API(s) can easily have a negative effect on user experience, the features which can be introduced and the cleanliness and maintainability of the developed store.
Summary
To summarize, speaking from experience, we find that Shopify is an awesome tool to do the job it has been built for but you’ll need to use some awful hacks if any advanced features are needed. In reality to achieve a half-decent store both approaches — template-based and API-based — need to be mixed closely together. However, the end result will most probably be still disappointing. You definitely shouldn’t choose Shopify as a technology to build any large or complex projects, it’s meant for the small, lean shops. This sounds simple but in reality people seem to forget this logic and try to pull off every possible feature using Shopify, building and using hacky plugins and promoting ugly workarounds that should never exist in production. Choose wisely.
