Are your Figma Design System analytics high as a kite?
Figma recently released a very welcome update to analytics, you can now get analytics data for styles and variables.
I was pretty excited about this. Reporting on token usage – whatever your flavour, fills a big old gap in reporting this side of the system.
Woah, we’re token millionaires
I was pretty blown away by the numbers of inserts and instances of styles and variables. The numbers were much bigger than I thought they would be – at times in the millions, but hey, this is the first time I’m seeing this data. Maybe this is normal.
Hang on
I noticed one of our variable libraries had usage in the hundreds of thousands and used by about 80 teams. Great!
Well, this library isn’t published to anyone. The variables in this file are used to build components that are distributed, but the variables themselves aren’t.
Dog with a bone
This began a 3 week long investigation. When I say 3 weeks, 5 minutes a day for 3 weeks.
I can’t ever let this kinda stuff go, I have to find out.
I really do apologise to Figma support for my daily emails. I’ll buy you all a coffee one day.
The question(s)
How can a library that isn’t published be recording inserts?
Can we trust the variable and style numbers?
As there is no way consumers can use this library directly, it had to be through interacting with the components that use them.
The process
To test this I had to set up multiple new Figma teams. One to author a couple of components and a few variables and a couple of other usage teams, suffixed with the date of the test.
I named everything in a way that would be easy to isolate.
Component A uses variables suffixed with “a” and not surprisingly Component B uses variables suffixed with “b”.
In the usage teams I inserted each component and ran a few tests.
For example, In one team I duplicated each component 3 times. In another I duplicated the components and detached a variable, clicked a variable, typed 99 bottles of beer into a variable. Any combination you can think of I tested it.
Because analytics only refreshes every day, any action I took I had to wait until the next morning to see the results…
It turns out variable analytics seemingly takes 3 days to come through when components are 1 day. This meant I ruined my first 3 days of tests and had to start over.
In the background I was still speaking to support and Figma experts to get some thoughts and insights.
The theories to disprove
For the last round of testing I decided to predict the results. I recorded the exact actions I made to Component A and Component B so given a few scenarios I could predict exactly what the analytics would read 3 days later. Sure enough, one of the predictions was bang on.
The results
So the answer to all of this is about what I am calling passive inserts. I called them that because the consumer doesn’t know they’ve inserted them. It’s simply happened to them.
This is of course the opposite to an active insert – where a consumer has inserted a variable by choice.
Drum roll please…
If a component that contains variables is detached, each variable that’s used counts as an insert, every time it’s duplicated.
Is that all I hear you ask? Well think about how many variables a component might contain? 8, 16, 24, 48?
Even if you have low detach rates these numbers can sky rocket.
If a single component that has 24 variables is detached and duplicated just 5 times (maybe deployment screens at 5 breakpoints) then that’s 120 inserts just from one basic component.
I have estimated this is causing hundreds of thousands of passive inserts across libraries, even if the detach data is below 1%.
So in answer to my other question – Can we trust the variable and style numbers? The answer is no, not really, not yet.
The problem is a passive inserts are not an insert, it cannot count towards a story of positive usage because it’s coming from the negative action of detaching.
It makes total sense that these variables would be captured by instances, they become real instances in use at the point of detaching. They were just never inserted.
Capturing the data
For now, we’re capturing all of the raw data anyway. But we have decided to report on only 5% of variable and style usage.
Unless we can determine active inserts from passive inserts then I would be worried about presenting insert numbers to anyone.
What has been fantastic is to see the instances data. This is going to be a really useful set of insights to have.
If we want to change that text colour, it’s good to know it’s currently in use in over a million places!
I’m hoping that we’ll be able to differentiate this usage in the future and make variable and style numbers a bigger part of our design system reporting.
