Dealing with Bugs in Scrum

One of the big questions we had to answer for our Scrum development process whether or not to story point bugs. Development wanted to story point bugs to help with sprint planning. The Product Owner (me) didn’t want to story point bugs to maintain an accurate velocity. We experimented with a few approaches and settled on one that works well for us. In this post I’ll run through the issues and solutions we came up with.

Our Problems with Bugs in Scrum

Bugs pose a bit of a problem in Scrum because of how velocity and scheduling work. The measure of progress is the amount of value added to the product, and we use story points to estimate how much of that value we can fit into a release. This is tracked as “velocity.” For developers, velocity also acts as a benchmark for deciding how many stories to include in any given sprint.

This use of velocity is basic Agile practice and works pretty well for the most part. However, bugs are going to happen no matter how good your developers are. As a Product Owner, I evaluate the bugs that come in and decide if/when to fix them. From my point of view, the bugs are scheduled just like stories… the most important things get done first.

When we get into sprint planning, Development has to figure out how much to pull into the sprint, including both stories and bugs. This is a little tougher because bugs don’t have story points, so you have to “groom” each bug during sprint planning to get a sense of how much work it requires. Sprint planning gets much more painful when it also involves grooming. You might argue that there shouldn’t be enough bugs to make this a big problem, and I wouldn’t disagree. But we’re transitioning an existing project to use Scrum, so we have a number of bugs from the B.S. (before Scrum) times. Even for new projects, sometimes bad things happen to good people and you end up with more bugs then you’d like.

Development requested that we start story pointing bugs to make sprint planning a bit easier. There are a number of reasons why I strongly suggest you DON’T do that, but we’ll leave that for another day. So, short of story pointing bugs, how can we make dealing with bugs easier in Scrum?

A Possible Solution: Bug Points

We experimented with a few solutions and settled on using both Story Points and Bug Points. A Bug Point is a Story Point, but only gets used for bugs and does not get counted in the velocity. We use the same scale as Story Points and assign them during our normal grooming sessions. Really the only difference is that we put them in a different field in JIRA (our issue tracker). Development also keeps track of “total work” for sprints, which includes both stories and bugs. However, this metric is NEVER used outside of sprint planning. At no point will I (the product owner) ever look at that metric and say “Well, our velocity is 15, but really we’re doing 20 points of total work, so I bet we can squeeze in these extra stories for the next release.”

When it comes to sprint planning, the team gets together and works out what can fit into the next sprint. The bugs in the backlog are prioritized along with the stories because some bugs need to be fixed now and others can wait for a while. Since bugs are also pointed, the team can easily look at the backlog and determine how far they can get. They don’t have to reevaluate each bug as it comes up in the backlog; they’ve already done that work and can just keep rolling.

This works well for us because it’s easier to know what’s going to fit in the sprint. It also allows the team to have a somewhat more realistic view of how much they can fit in a particular sprint. Note that I said a particular sprint, not a release. Remember:  NEVER consider bug points during release planning! If the team has bugs in 4 straight sprints, then don’t have any they have a better idea of how much they can fit without any bugs to work on.

There are a number of other benefits. For one, you get a better sense of how much extra work you’re creating by letting bugs slip through the cracks when implementing a story. This helps to spot quality issues. If you’re constantly creating more bugs than value then you know something is wrong.

What’s Not So Good About This?

It’s very easy to fall into the trap of using bug points like story points outside of sprint planning. As I mentioned earlier, including bugs in your long term planning is a bad idea. Bug points are so similar to story points that it’ easy to forget that they AREN’T. Other than that, this seems like a pretty good solution to me.

It works for us, but how about you? I’d love to hear about how others are handling bugs in the backlog.

It’s Hadoop’s world. We just live in it

It’s Hadoop’s world. We just live in it. Welcome to #hw2011!

That was the starting battle-cry from Mike Olsen, CEO of Cloudera as he kicked off the third Hadoop World Conference. Indeed, after drinking the kool-aid for two days, I’ve been almost fully ingested, stored and transformed even if I have yet to be accessed, let alone managed.

For it seems that within a few years, all digital information, including my electronic Freudian id and perhaps my ego as well, will be deposited into Hadoop, forever ready to be accessed via any number of different social and structured graphs.

Hadoop background

Some have speculated that within five years, Hadoop will hold 50% of the world’s information. I now believe that to be true, albeit potentially a copy of the other 50%, if not uniquely in Hadoop.

Hadoop and its Google ancestors enable storage on a scale and scope never previously possible with baked-in redundancy and resiliency at lower operational cost than big iron solutions. And the software is free, needing nothing more common commodity hardware and a Java host.

Google created and shared the concept. Doug Cutting and a colleague started an independent Apache licensed implementation five years ago. Since then, it has been adopted by the largest web properties: Facebook, Twitter, eBay among others. Even major enterprises like JP Morgan and Disney have been using it in production for at least two years.

Commercially supported releases are available from Cloudera and Hortonworks.

The Conference

Hadoop is still in the early adopter stage and has not yet crossed Geoff Moore’s Chasm. This is most reminiscent of the state of the web circa 1994. Forward looking companies are making incredible strides in competitive advantage using primitive tools and smart developers.

Cloudera is doing great job in championing the ecosystem. They recognize that growing the overall market and adoption is the correct long term path to riches. I look forward to #hw2012.

NEUGS Part 11: Workflows, AKA Lifesavers for the Lazy

Not going to lie, guys, I’ve been putting this post off for a while. (Ironic, as procrastination is exactly what Workflows aim to prevent.) To me, the term connotates TPS reports and dingy cubicles and unsheathed florescent overhead lights and perpetually sweaty officeworkers in greasy button-downs and Bluetooth headsets. Also, blandly enthusiastic sales execs talking about connection and knowledge share and  koi ponds, though I’m not sure where that last image comes from. Butttt, here’s the thing: workflows provide a pretty useful method for keeping individuals and teams on track, through a series of automated steps triggered by the initialization or completion of a designated action.

Eg let’s say, completely hypothetically, that I am a fairly low-ranking business analyst at Kibble ‘n’ Krunchy Bits Corp. Let’s also say, again completely hypothetically, that I have this habit of uploading my weekly sales report to the sales team site, then wandering off to gchat for hours. So the reports just sit there without anyone looking at them for like, weeks at a time, and then at the end of the quarter everyone is surprised by how much sales of KrunchExtreme Lite with Passionfruit Extract ™ have grown. (Even though, hello, they should have known this because the factory workers and delivery men have all been putting in on average 13 hours of overtime a week for the past four months, figures which someone in a different department really should be keeping a better eye on.) A Workflow – in this case a modified Approval Workflow – provides me and my managers with an easy solution to this lack-of-awareness problem. Continue reading NEUGS Part 11: Workflows, AKA Lifesavers for the Lazy

A Simple Bash Script to Clone a Drupal Core GIT Repository

This is a simple bash script used to clone a Drupal Core GIT repository locally when passed a branch number and a new directory name (to host the Drupal repo). I use the issue number as the directory to keep Drupal API documentation patches coordinated with the Drupal.org issue number.

To execute the script, copy and past the commands in the code block into a text editor then save the script as filename.sh (e.g. clonerepo.sh).

Run from a bash prompt (in the same directory) by typing: sh clonerepo.sh

No more row limitations: Exporting XLSX with SQL Server 2008 R2 Reporting Services

The hard-at-work folks on the SQL Server Reporting Services team recently announced that SQL Sever 2012(formerly codenamed “Denali”) will feature the ability to export reports to XLSX and DOCX. These new OpenXML-based file formats were introduced in Microsoft Office 2007, and they contain a number of interesting features and capabilities beyond the older, binary XLS and DOC files.

But, if you can’t wait for the SQL Server 2012 release (expected to be released in the first quarter of 2012) to get XLSX and DOCX from Reporting Services, you can use OfficeWriter.

No more 65,536 row limitation!

This is probably the biggest reason that people want XLSX instead of XLS. XLSX supports over a million rows.

What is OfficeWriter?

OfficeWriter is a set of tools for Excel and Word document generation, manipulation, and reading. OfficeWriter consists of three main pieces:

1. OfficeWriter API – For .NET developers who need to build custom Excel and Word document generation, manipulation, and reading into custom applications.
2. OfficeWriter SharePoint Solutions – Packaged, no-code solutions for SharePoint that add document generation capabilities to SharePoint lists
3. OfficeWriter for Reporting Services – Custom extensions to SSRS that allow you to publish, populate, and deliver Excel and Word reports from the report server.

The last option is what I will focus on. Specifically, how to generate XLSX reports from SSRS.

How to Generate XLSX from SQL Server 2008 R2 Reporting Services

With OfficeWriter for Reporting Services, you can publish Excel and Word templates to the report server.  When the report executes at runtime, SSRS merges the Excel or Word template with data and delivers it to the client.

First, I’ll open Excel and flip to the Add-Ins tab where the OfficeWriter toolbar resides:

You can either build a new DataSet  for the report by clicking the Add Query button, or import a DataSet from an existing RDL by clicking Open Report.  I have an RDL with a specific query already defined in it that shows all 121,317 rows from the AdventureWorks 2008 R2 sample database.  So, I’ll click Open Report and just use that.  Once I open that RDL, the DataSets and all fields from those DataSets will appear in dropdown menus:

The RDL I opened has only one DataSet called TerritorySalesLineItems, and two fields called Price and TerritoryName.  To build the report, add any headers or styles you want using regular Excel formatting techniques, and then drop data markers into cells where you want data to go by clicking on items in the Insert Field dropdown:

Publish this workbook to the report server with the Publish button:

When it’s published, it will appear just like a regular report which can be secured, managed, and delivered as you would any conventional RDL:

To export the report, select the Excel designed by OfficeWriter option:

The output generated is the template merged with the data by OfficeWriter on the report server.  Note that the exported Excel has well over 100k rows:

Conclusion

Using OfficeWriter and Reporting Services together lets you pump a huge amount of data into Excel workbooks in a high-performance, highly scalable manner while preserving all Excel formatting and leveraging the features of the report server.

How to Generate XLSX from SQL Server 2008 R2 Reporting Services

With OfficeWriter for Reporting Services, you can publish Excel and Word templates to the report server.  When the report executes at runtime, SSRS merges the Excel or Word template with data and delivers it to the client.

First, I’ll open Excel and flip to the Add-Ins tab where the OfficeWriter toolbar resides:

You can either build a new DataSet  for the report by clicking the Add Query button, or import a DataSet from an existing RDL by clicking Open Report.  I have an RDL with a specific query already defined in it that shows all 121,317 rows from the AdventureWorks 2008 R2 sample database.  So, I’ll click Open Report and just use that.  Once I open that RDL, the DataSets and all fields from those DataSets will appear in dropdown menus:

The RDL I opened has only one DataSet called TerritorySalesLineItems, and two fields called Price and TerritoryName.  To build the report, add any headers or styles you want using regular Excel formatting techniques, and then drop data markers into cells where you want data to go by clicking on items in the Insert Field dropdown:

Publish this workbook to the report server with the Publish button:

When it’s published, it will appear just like a regular report which can be secured, managed, and delivered as you would any conventional RDL:

To export the report, select the Excel designed by OfficeWriter option:

The output generated is the template merged with the data by OfficeWriter on the report server.  Note that the exported Excel has well over 100k rows:

Conclusion

Using OfficeWriter and Reporting Services together lets you pump a huge amount of data into Excel workbooks in a high-performance, highly scalable manner while preserving all Excel formatting and leveraging the features of the report server.

Get started:

Learn more about SSRS integration or start your free trial of OfficeWriter today.

      OR      



Stories from the WIT Trenches: Debra Dalgleish

[This is the sixth in a series of posts exploring the personal stories of real women in technology. Every woman in tech overcame at the very last statistical odds to be here; this blog series aims to find out why, and what they found along the way. If you’ve ever sought Excel-related help online or in print, chances are you’re familiar with Debra Dalgleish, one of the foremost authorities on Excel and Access development and the author of three books on pivot tables. Here, she talks staring your own home business, getting young girls aware of and excited about careers in tech and kissing correction fluid goodbye. If reading her story inspires you to share yours, please feel to email me.]

Self-employment is the dream job – most of the time. As a computer consultant, working from home, you can set your own hours, schedule meetings at convenient times, or meet with clients online. You’re the boss, so you can pass on projects that don’t appeal to you, if your workload gets too high.

If you have a young family, running your own computer-based business can give you more time with the children, while still earning an income. That’s why I got started, and now, even though the children have moved out, I wouldn’t want any other job.

My work, as an Excel and Access developer, is challenging and rewarding. My clients bring interesting projects, and push me to continue to improve my skills. In this business, there’s always something new to learn. Continue reading Stories from the WIT Trenches: Debra Dalgleish

Boston SharePoint Salon Recap: Powerview and Fall in a Glass

Cross-posted from bostonsharepointsalon.com:

Last night’s Salon was, by I want to say all accounts, a rather smashing success. (Perhaps a smashed success for a few philosophes.) About twenty locals, newbies and visitors braved the seriously odious weather to talk about, among a few many other things, Power View and the revamped BI Stack. Many many thanks to Sean Boon (b | t), from Microsoft’s Power View team, for coming all the way from Providence on a tweet’s notice—your inside expertise was much appreciated!

If you’d like to come to the next Salon, consider this your invitation! It’ll happen sometime in mid-December–stay tuned for specifics!

Everything You Wanted To Know About Power View—But Were Afraid to Ask

Before you decide whether Power View is the best damn thing to happen to self-service BI since graph paper or is just a smoke and mirrors, CamelCaseless extension to PowerPivot, you need to know its gist. The following blog posts and videos will give you just that, from a (mostly) business user perspective. Read ’em, and then get cracking with the CTP3 version, available for download here.

  • Dan English’s (b | t) “Intro to BI Semantic Model & Delivering Self-Service Reporting with Power View (Crescent)” video and slide deck

A comprehensive MSBI presentation that covers the BI Semantic Model concept, Power View and SQL Server Analysis Services with Power Pivot in SQL Server 2012.

Continue reading Everything You Wanted To Know About Power View—But Were Afraid to Ask

Blogged