Using Agile to Help Fix Big Data s Big Problem

Using Agile To Help Fix Big Data S Big Problem-Free PDF

  • Date:29 Nov 2019
  • Views:51
  • Downloads:0
  • Pages:6
  • Size:243.09 KB

Share Pdf : Using Agile To Help Fix Big Data S Big Problem

Download and Preview : Using Agile To Help Fix Big Data S Big Problem


Report CopyRight/DMCA Form For : Using Agile To Help Fix Big Data S Big Problem


Transcription:

Big Data s Big Problem and the es people spend more time sitting in meet. Agile Solution ings and managing handoffs than they do. At their best big data analytics detect pat on actual data related activities The work. terns that would require considerably more itself may be misdirected and wasteful The. time and effort to uncover using traditional final product is often late and difficult for a. analytics tools The widespread use of big lay executive to understand and the impact. data analytics has powered breakthroughs is less than anticipated. in areas as varied as medical diagnostics, people management and the ways organi Frustrations with the waterfall method. zations respond to consumer behavior eventually led software developers to im. prove the process by adopting agile ways of, Data scientists start big data projects with working Agile calls for working in a way. a theory about a business problem such as that is iterative empirical cross functional. how to predict demand for a car model on focused and continually improving See. the basis of new features and past sales or Five Secrets to Scaling Up Agile BCG ar. how to determine how many employees an ticle February 2016 Common agile meth. organization should hire to staff a new ven ods include assembling cross functional. ture given existing personnel and previous teams which improve communications and. staffing levels for similar projects They reduce handoffs especially when team. then build algorithms to test the theory us members work in the same location They. ing one or more forms of artificial intelli also include developing minimum viable. gence machine learning optimization or products MVPs rapid updates and fre. traditional statistics on a massive scale If quent feedback to ensure that the finished. the theory is shown to be false they may product delivers on expectations and goals. continue refining and testing it until they See Exhibit 1. reach a valid conclusion Or they may drop,it and move to a different or more pressing. business problem Using Agile in Big Data Projects,Agile has shown great promise in many. The results of big data analytics can be re fields besides software including financial. markable In observing many companies services marketing and consumer goods. though we have also seen significant failure See Agile to the Rescue in Retail BCG. rates particularly when organizations at article October 2018 and Taking Agile. tempt to roll out those analytics on a wide Transformations Beyond the Tipping. scale When problems do occur deliverables Point BCG article August 2018 In client. such as any expected insights or process im engagements we have seen agile empower. provements may not materialize More of teams to do their best work while assuring. ten than not the fault lies not with the data that they are aligned with an organiza. but with the methods used to verify pro tion s strategic goals Given those successes. cess and act on it See How to Avoid the we believe that agile could bring several. Big Bad Data Trap BCG article June 2015 specific benefits to big data projects. The heart of the problem is the manner in Rapid Experimentation Historically. which big data analytics are developed testing occurs near the end of big data. Most are built sequentially applying the projects which means that business. waterfall method of project management executives might not see results until then. traditionally used in software development For example a team building a predictive. In the waterfall method data scientists ac analytics model to help sales people. quire verify and integrate data develop a convert leads might wait until late stage. model or algorithm to test it run the test testing to show executives the results. and then either act on the results or contin However offering results so late in the. ue refining the model Work on one task process could lead to unclear expectations. waits until the preceding task is finished for the work to be done methods to be. But that process is inefficient In many cas used and possible outcomes With agile. The Boston Consulting Group Using Agile to Help Fix Big Data s Big Problem 2. Exhibit 1 Agile Speeds Up Traditional Big Data Projects and Improves Outcomes. Traditional,Long time to market,Teams work in silos with.
multiple hando s,Requirements Design Development Test Corrections. Lack of focus,end product Limited emphasis on,Customer need delivering business value. Agile Faster product design,Additional,Test MVP Frequent iterations to. Corrections MVP MVP customer deliver an MVP, Develop customer customer feedback Small focused cross. feedback feedback as needed,functional teams aligned.
Design with business goals,Requirements,Strong emphasis on. Customer need Sprint 1 Sprint 2 Sprint s 3 continuous improvement. and delivering business,Source BCG analysis,Note MVP minimum viable product. projects are broken down into manageable big data project MVP includes the business. chunks that can be built and tested quickly and behavioral changes needed to attain. Teams develop and test MVPs continuous real results For example developing an al. ly If the data analytics do not yield the gorithm to improve scheduling and dispatch. expected results business executives find for service technicians could also include. out right away and can correct their course developing a different process for notifying. They could ask the team to analyze differ customers of upcoming appointments. ent data make other modifications or, even in some cases abandon the project Early Customer Feedback The overriding. all moves that save time and money goal of big data projects is not to build. compared with other methods brilliant mathematical models but to solve. practical business challenges or discover, A specialty retailer took this concept of rap insights leading to actions that could benefit. id experimentation to heart when it con customers That makes it important to. vened an agile team that adopted the mot include customers in the process If a project. to Get 1 better each week The agile is for an external client a representative of. team which was composed of personnel the client could be embedded with the. from the data engineering data science team If a team is working on a big data. marketing and creative functions among project for an internal client a member of. others was tasked with developing inno that department might be on the team. vative big data marketing and sales solu When a European oil refinery created a big. tions To do that they ran agile sprints that data application that its engineers could use. produced incrementally new omnichannel to optimize the maintenance cycle of their. marketing programs every seven days The key equipment for instance some process. new programs led to direct increases in rev and maintenance engineers were assigned. enue in a short period of time to the teams that developed the app. Rapid experimentation is appropriate not Prioritizing Value Accomplishments that. only for big data analytics algorithms and add value without increasing cost take. the data that a project is based on but also precedence over completing tasks in a. to ensure that an organization can under predefined order If the team determines. stand and act on the results For this reason that including a particular feature will take. in addition to the algorithm and output a significantly longer than expected without. The Boston Consulting Group Using Agile to Help Fix Big Data s Big Problem 3. providing a lot of additional value the nel from such key functions as operations. product owner the team member respon technology development asset mainte. sible for representing the customer can nance and IT Data scientists from an out. take this into account The product owner side advanced analytics consulting firm. can drop it and move on to lower cost or were included on the teams The teams. higher value items on the project backlog a and agile approach helped the refinery. prioritized list of work items produce multiple MVPs within a four. month span and industrialize a final prod, Cross Functionality Traditional big data uct considerably faster than it could have.
projects fail most often for reasons that are done in the company s typical product de. largely unrelated to data analysis In our velopment cycle Multidisciplinary scrum. experience working with clients 70 of a teams also contributed to a more collabora. cross functional team s efforts reach tive corporate culture and drastically in. beyond strict analytics into the business creased employee engagement. processes operational behaviors and types, of decision making that the analytics People Empowerment In a traditional big. suggest To accommodate that scope big data project a project manager decides. data project teams typically include which priorities are most important and. personnel with a variety of backgrounds how they will be met even though he or. See Exhibit 2 The team can make deci she may not understand the development. sions without members needing approval process When that power is delegated to a. from their individual bosses Team mem team people become more engaged in. bers work out tradeoffs conflicts and their work and are more invested in the. compromises in real time which explains outcome Unlike data scientists who might. why it is so critical for them to work in the work on multiple engagements for exam. same location preferably in the same ple agile team members are not staffed on. room At the same time that they are several projects simultaneously Rather. working on algorithms and data teams they devote all their time to the team thus. may also be making changes to operating becoming more invested in the work This. models and business processes singular focus also builds accountability. The European oil refinery mentioned It s no wonder that companies that have. above took such an approach when it in adopted agile are more successful than oth. corporated agile ways of working into three ers in attracting digital talent and younger. big data projects launched as part of a larg workers two groups of people who priori. er digital transformation initiative Each tize work that gives them a sense of pur. project had a dedicated scrum team with a pose See How to Gain and Develop Digital. scrum master product owner and person Talent and Skills BCG Focus July 2017. Exhibit 2 Agile Teams for Big Data Projects Have a Cross Section of Expertise. Product Asset Technology Data Data Outside, owner management Operations development Planning engineering Design science consultant. Agile coach project 2,Source BCG analysis, The Boston Consulting Group Using Agile to Help Fix Big Data s Big Problem 4. The Secrets to Using Agile may indicate signs of progress toward a. in Big Data satisfactory solution Accepting such, Agile and big data may sound like a perfect imperfection may require stakeholders. pairing but getting them to work together who in the past saw only near final. is not as easy as it may seem To be success versions to shift how they think about big. ful keep several critical factors in mind data analytics projects By encouraging. trial and error stakeholders improve the, The algorithm is not the finished product odds that a project will move successfully.
An algorithm can be a thing of beauty or a from MVP to full scale production. waste if it cannot deliver results oriented, output in a way that a business depart Include more than just data science. ment or organization can understand Agile personnel Agile teams should include a. teams should develop dashboards info mix of talent assembled on the basis of. graphics or other visuals that readily need rather than on a standard structure. communicate the results of their findings or past experience As a general rule it. They must also include as part of their makes sense to staff a project team with. work the end to end operational and data engineers who can prepare the data. behavioral changes that are necessary to data scientists who can conduct the analy. get real results sis designers who know how to present the. data and a variety of business personnel, MVPs are distinct from prototypes Proto who are familiar with the project s busi. types come first and generally are built ness objectives and implications for. with historical data in order to verify that existing processes The goal is to blend. an algorithm can do what it is supposed to people s talents into a whole that is greater. do If a prototype works it is used as the than the sum of its parts. frame for a more all encompassing MVP,that could potentially be released to end. users An MVP also includes up to date,data a user friendly interface core fea. tures and operating instructions And since,W ith so many positives to be gained.
from incorporating agile into big data,projects companies might be tempted to. it has to work in a business context it dive in right away But getting the best out. includes relevant changes to processes and comes takes substantial planning including. operating models as well a thorough examination of what is needed. and how it would affect existing personnel, Stakeholders must accept imperfection and processes Conducting a pilot is a good. The iterative nature of agile development way to start If it succeeds agile can be add. brilliant mathematical models but to solve practical business challenges or discover insights leading to actions that could benefit customers That makes it important to include customers in the process If a project is for an external client a representative of the client could be embedded with the team If a team is working on a big data

Related Books