TL;DR: So, your company decided to outsource development process to an external software house and you’re responsible for its success, right? Below you find some advice from the trenches – do’s and don’ts. For those who work in SCRUM but not only. Let’s go!
First – be clear in requirements but leave some space in budget and schedule
Working remotely with your IT team requires precise communication and requirement gathering. Is Ok to use simple tools like paper wire frames or describing user stories in simple form in JIRA / Confluence (like “As I user I would like to be able to change order status and push it to ERP”).
Critical here is to write down Acceptance Criteria in each user story. That means – measurable criteria if the feature is to be accepted. Such as: “After switching the status to ‘paid’, the order should be visible in ERP in ‘incoming orders’ tab after no longer than 10 minutes with all the details I see in the web shop – like addresses and order lines”. Simple but powerful. You shouldn’t describe how (exactly) UI should look like, but of course you can give some advice here. You should invest some time in designing – or order UI design to have an exact view of the application. But this is usually the next step.
You have to resist the temptation of small improvements. What do I mean by that? Let me give you a short example: changing some details in graphics (based on wire frames) don’t expect developers to implement both features on wire frames and graphics. They will implement graphics because it’s the last accepted artifact. Be precise, be consistent – it saves your time and money.
Developers and people at your company are just people, so be aware that you have to be ready for two things:
- you have to consult all requirements with users at your company who are to use this feature and get their acceptance; commitment and engagement will be crucial later on when system is ready to deploy; you are the PO and you always have the last voice, but your role is to gather all points of view in one place and be the single point of contact to with the IT team; be in charge of how the system looks like; never avoid this responsibility or you are bound to fail
- no matter how precisely you describe requirements and consult them with your business stakeholders, there will always be changes. Some requirements can be misinterpreted by developers, some can be described more precise in your imagination than in writing :) don’t blame the team at first place, leave some space in budget for corrections and changes.
After all, you’ll be ready to decide whether to accept a feature (or not) after clicking through it. So…
… test often, test early
Yeah, this advice is kind of evergreen – but – I assure you that it’s easy to underestimate the time it needs to be done well.
You should test a system after each demo (let’s say in 2 week sprints?). I think you need to reserve half a day or more to do so. Test each ticket, acceptance scenarios, give feedback.
On this stage the system will contain lots of issues, it will look all wrong. But it should work in essence. Be eager to give quick feedback because it all shortens the final UATs (User Acceptance Tests) and confronts your imagination with the real application. Don’t assume it will work differently later on or so – check everything you require from the final solution and point out any issues to the team. They will do the work.
How many issues is it ok to have after a demo? It depends – 20/40/100? Really, it depends on a given sprint scope and how much time you invest in testing. More tests = more issues.
Divante isn’t now able to engage me in each project tests. But I’m trying to test every application we’re to publish. Another pair of eyeballs always looks at some new issues. 20-40 issues after such tests? Normal situation in a well-tested project.
So you’ll ask what are testers supposed to do? They cooperate with developers at this stage, testing the behavior of each requirement, retesting after changes and writing down testing scenarios for UATs. They are just people – they aren’t you or your stakeholders. They can misinterpret something, maybe miss something. Don’t expect an ideal system. You’re not buying a readymade product but a dedicated IT solution. There is always something to be changed.
If you’re not the only business stakeholder of the final user (a rare situation :)) you should also invite some other employees from your company to do the tests. But – not too early.
Maybe it’s good idea to invite one person from each team that is to use the application and let them confront their expectation? Doing so, you can improve UI or apply some critical business improvements that wouldn’t be discovered elsewhere – when there is time to do so.
Involving too many business users too early can damage the system image as “buggy”, “useless” and so on. So be careful. This can damage the whole system adoption when done wrong. Or if it haven’t been done at all.
Application is ready … when it’s ready.
Ok, it can sound a bit strange. what do I mean is when your project is ready for Acceptance Tests (UATs), it won’t necessarily mean it’s ready for deployment. It’s a rare situation and its strongly influenced by how much effort you had put in before this stage.
If you don’t want to fail your stakeholders’ expectations, you should plan at least 2 weeks full of tests and then 3-4 weeks of bug fixing and retests.
I assume of course you were involved in tests before and there no huge revolutions should be expected. You should involve business users on this stage as well.
Please expect 180/200 issues as an average result of UATs on medium eCommerce. It’s normal. The team should handle these numbers in 2-3 weeks, because most of the time the issues are easy to fix UI ones.
What’s important here: please remember to reserve your internal IT team for tests as well. Usually some sophisticated integrations like WMS, ERP need to be tested before UATs. It can take even more than two weeks. They have to be online with remote team, have required time reserved to test systems on both sides.
They should go through test scenarios (written before tests) and test end-to-end processes: like from making an order, to reserving WMS stocks and billing an invoice in ERP.
After internal tests and fixes you should plan some pre-launch / soft-launch stage. From my perspective, it was always a sign of business success when this kind of preparation was planned.
What I mean by the above: launch for a limited number of users, like selected companies or customers (or 10000 customers selected from 1 million). They will give you feedback; you can also get some incentives from them. Two – four weeks of soft launch is a good choice in most cases.
How to communicate and with whom?
More communication is always better than less, but be careful. From my experience, business holders should communicate with business holders, not directly with IT people most of the time, or have translators :) A business holder is Product Owner on your side and Project Manager on IT company’s side. Who can be the translator? An IT consultant from a third party or someone like that.
Why? On a daily basis there are sometimes stressful issues to be solved, like deadlines, scope bickering … Sure, SCRUM suggests that all the team should be involved in communication but please notice two things:
- you should communicate with the team if it’s about scope, features orfeedback,
- solve other issues with the PM, so that you don’t demotivate and distract the team. They are often strong in IT but not in interpersonal communication. They can be easily demotivated and overwhelmed.
SCRUM daily meetings are not to solve this kind of issues. They are to catch up with progress, listen more than speak. Remember this and don’t provide distraction and don’t waste motivation and time of developers!
Ok, this entry is getting top long. If you managed to find yourself here: congratulations and thank you :) Feel free to exchange your own experiences in comments or via pkarwatka@divante.pl