Introducing a product supply tradition at Etsy

Deciding on a associate

Etsy had by no means labored with a software program growth consultancy. To find a way
to scale product supply and usher in some outdoors experience, they began
on the lookout for a vendor. After an intensive search, they chose Thoughtworks,
primarily as a result of they perceived a detailed cultural match and deep alignment
relating to trendy software program growth.

Thoughtworks has an analogous strategy to Agile as Etsy – being Agile
relatively than doing Agile. Etsy’s group was not seeking to undertake a
methodology that blindly follows strict rituals and practices, and wished
a associate that bases its strategy on agile ideas and tradition.
Thoughtworks is thought for technical excellence and management in DevOps,
Steady Integration, and Steady Supply, all of which Etsy depends
on closely. As well as, the 2 firms shared comparable ideas
relating to worker care, variety, open-source, and know-how
sustainability.

The Thoughtworks group began by embedding into product groups, working with
each purchaser and vendor groups, on varied essential initiatives corresponding to cost
mannequin adjustments, Search Engine Optimization (search engine optimization), and a notification platform.
This enabled Thoughtworks to grasp how Etsy labored, created an instantaneous
affect on supply, and improved growth practices corresponding to automated
testing.

For Thoughtworks, Etsy was fairly completely different from the everyday consumer.
They’re a completely digital firm, and their technical course of and product
practises are very mature. In contrast to most purchasers who have interaction Thoughtworks,
Etsy had no massive transformation to undertake, and no ingrained habits
that wanted a serious shift. What they have been searching for was outdoors perspective
and supplementive expertise, to assist them incrementally enhance and sort out
their urgent initiatives. The tradition and ideas have been already very
aligned with Thoughtworks’ personal, so the partnership grew to become extra of an
alternate of concepts drawn from completely different experiences.

Scaling Product Supply and Discovery

Thoughtworks and Etsy shaped a cross-functional Product Supply
Tradition (PDC) group to research the present product supply and discovery
course of. They analyzed the complete software program growth worth stream,
trying carefully on the complaints of managers and group members. Chosen
occasions have been replayed to achieve a deep understanding of “how” it occurred
rather than who?. The Thoughtworks
group, as a result of they hadn’t lived by means of the issues, have been capable of deliver
an goal outsider perspective.

There have been quite a lot of issues Etsy was doing very well:

• Cross-functional groups: They structured their groups round “4 desk legs”:
  Product, Design, Engineering, and Analytics. All planning and supply practices
  occur with collaboration among the many teams.
• Incremental supply of worth to customers: Steady supply is the core
  observe of their strategy. A code change sitting in a repository and never
  deployed to customers is just not precious, and successfully prices the corporate cash. Etsy
  deploys a whole bunch of instances a day, with a 30-minute decide to manufacturing lead
  time.
• Options are prioritized utilizing information: Product managers and information analysts
  collaborate to find out an anticipated worth, measured by KPI adjustments. If the
  function doesn’t ship sufficient worth versus its value, as evidenced by an A/B
  take a look at, it isn’t rolled out, and the code is eliminated.

The group additionally uncovered alternatives for enchancment:

• Every group was doing supply in numerous methods, having disparate processes
  and rituals, making onboarding new workers and altering groups tough. Some
  groups have been utilizing a course of like Scrum, whereas others didn’t use any formal
  course of.
• There have been quite a lot of initiatives that had not created worth as
  they anticipated. For instance, Etsy tried a present wrap function that had
  decrease than anticipated adoption charges by sellers and patrons.
• Previous to 2018, Etsy had been by means of a transitional interval, with
  pivots in product technique and management adjustments main some product
  managers to expres a want for higher possession over their product
  choices.

Time to studying

The group created an enchancment program, based mostly on lean pondering and the
concepts of Marty Cagan, who had not too long ago offered at Etsy. The management group was
studying his e book Impressed.

I promise you that a minimum of half the concepts
in your roadmap are usually not going to ship what you hope. (By the best way, the
actually good groups assume that a minimum of three quarters of the concepts gained’t
carry out like they hope.)

— Marty Cagan, Inspired (p19)

After some preliminary analysis the group got here up with a metric they known as Time
to Studying – the time it took for a product group to validate an concept with a
buyer and acquire a greater understanding of its worth. That they had a baseline of
50 days that they wished to cut back.

They have been additionally different metrics.

• Enhancing the time to market
• Enhancing the effectiveness of merchandise (e.g. income, buyer
  engagement)
• Worker happiness

To affect the KPIs, they got here up with quite a lot of resolution hypotheses:

Lightweight prototypes that incorporate extra direct person analysis

Etsy has robust experimentation infrastructure and analytics capabilities.
They run many concurrent A/B exams on {the marketplace}. The issue the PDC
group noticed is that the suggestions cycle could be fairly sluggish. For an concept to be
prepared for an A/B take a look at it needed to be close to production-level high quality. It additionally had
to have sufficient information to point out statistical relevance. A/B exams would run for
months in lesser-used components of {the marketplace}.

To set your expectations, robust groups usually
take a look at many product concepts every week—on the order of 10 to twenty or extra per week. I
need to emphasize that these are experiments, sometimes run utilizing prototypes.
A prototype is just not one thing that’s prepared for prime time and positively not
one thing your organization would attempt to promote and stand behind. However they’re
immensely helpful, as they’re all about studying quick and low cost.

— Marty Cagan, Inspired (p27)

To cut back the Time to Studying metric, the PDC group utilized a lean UX
strategy. First, they began by rapidly creating lo-fi prototypes and
instantly exhibiting them to customers. Second, the group elevated person suggestions
classes to each two weeks and lowered the extent of ceremony. Third, they
created a dual-track system, with a design and analysis group regularly
working forward of a supply group. The concept was to effectively acquire confidence
in an concept earlier than it qualifies for an A/B take a look at. By decreasing the price of
experimentation, they might improve the range and amount of experiments,
discovering concepts that confirmed extra worth.

Determine 1: Experiment validation course of

Blueprint for product supply

The PDC, at the side of product group leaders, got here up with a set of
product supply ideas and associated supply observe adjustments. The
practices included: smaller extra centered tales, redesigning story partitions to
higher match the group course of, utilizing spikes to cut back technical uncertainty, and
higher methods of managing tech debt. They didn’t need to take away the autonomy of
groups. Steady enchancment is a giant a part of Etsy tradition. These practices
are supposed to be handled as smart defaults that may be tailored and improved
for every group’s particular context.

The PDC group first examined the observe adjustments on pilot product groups to
set up their effectiveness after which iterated. As soon as the observe
demonstrated an enchancment, the group added to a data base and supplied
gentle teaching to introduce the brand new practices to all product groups throughout
Etsy.

Lean Portfolio administration

Some massive initiatives have been nonetheless being carried out, regardless that there was
not clear proof of the initiative creating worth. That is the everyday sunk
value drawback. To resolve this, the PDC group
recommended taking a lean strategy to portfolio administration.

Our runway ought to be a listing of hypotheses to check, not a listing of
necessities to construct. After we reward our groups for his or her means to ship
necessities, it’s simple to quickly bloat our merchandise with pointless
options—resulting in elevated complexity, increased upkeep prices, and
restricted means to alter. Options delivered are usually not a measure of success,
enterprise outcomes are. Our runway is a sequence of questions that we have to
take a look at to cut back uncertainty and enhance our understanding of development
alternatives.

— Humble et al, Lean Enterprise (p96)

As a substitute of making prolonged mission specs and plans, the PDC group
coached product groups to develop end result statements for every initiative. With
end result statements, beginning with a buyer or enterprise end result permits the
group to experiment with many alternative concepts to realize the end result by tying
them to measurable KPIs. The much less exact an end result is, together with its KPIs,
the tougher it’s to specify. When making ready for doable initiatives, every
product group created a two-page transient summarizing the end result, the measurable
affect, and the preliminary funding required.

The chief group prioritized the end result statements, and every assertion
was considered as a “guess” amongst a portfolio. Impressed by the Lean Worth Tree
idea, the manager group would observe an everyday cadence to find out if a
guess was paying off, giving them the chance to pivot early.

Stronger product & engineering collaboration

One other enchancment was to incorporate engineers early within the product ideation
course of and to have them be part of person suggestions classes. The group examined
engineers becoming a member of the speculation era and validation classes and
working carefully with designers throughout design sprints. The outcomes of the take a look at
confirmed a number of advantages. Engineers improved their understanding of the
buyer, constructing empathy for his or her wants. They may present on-the-spot
suggestions on the feasibility of an strategy and rapidly create prototypes to
reveal concepts to customers. Cross-functional engagement allowed engineers to
contribute quite a lot of nice concepts that made it to the product.

After the preliminary PDC initiative Etsy continued to strengthen the
collaboration between teams. At each stage of the group, product and
engineering are companions, which suggests they’re capable of have wholesome trade-off
discussions. The platform group expanded using product managers for
infrastructure and developer tooling groups – encouraging groups to consider
technical capabilities as merchandise and end-users being Etsy engineers. The
technical product managers do analysis and prioritization actions, simply as
they might for a buyer product.

What was the affect of product supply tradition?

Testing with pilot groups indicated that the Time to Studying went from 50
days to five days. In the course of the pilot, one product supervisor stated, “What I like about
the [direct user research sessions] pilot is that it lets us take a look at our concepts
quicker, it was very nice to not spend as a lot time diving deep into
one thing that we didn’t even know we have been going to make use of.” An initiative on
stock high quality that was predicted to ship a seven million greenback income
improve as an alternative delivered eleven million {dollars} in incremental worth utilizing
the PDC recommended enhancements. Following this pilot success, the group coached
the complete product growth group on the product supply strategy.

In response to Mike Fisher, “It is extremely onerous to isolate the advantages of a
course of change on a company of a number of hundred engineers since we don’t
run A/B experiments of this sort. Nevertheless, we did have a look at excessive stage
productiveness metrics after having PDC carried out for over 2 years, and all of
our metrics have been very constructive. We had improved our velocity by doing this
work, and we have been capable of produce in the end extra advantages for the patrons and
sellers – extra options, higher search, higher processes; the entire
market grew to become higher for patrons and sellers and that was the objective.”