Many CIOs have faced the challenge of how to align their technology investments with overall corporate strategy. eBay, the world's largest e-commerce auction site, faced a different type of challenge. From day one,  eBay had to create its own world to be successful. Technology plays an integral part in shaping the direction and the growth of eBay's business. One could favorably argue that eBay is about technology. Because of its business model, eBay had to build its architecture from scratch and to devise new best practices to support it. James Barrese, eBay's vice president systems and architecture, says that building many key components can create new problems that must be addressed without the legacy safety net to fall back on. enterpriseleadership.org recently sat down with Barrese to talk about how his group maintains eBay's agile architecture and scalability to support a 223 million user base, which grows by 130,000 new users each day. Here's what he had to say:

EL: Can you describe what your organization  does?

JB: The architecture team works on the OS for eBay. You can imagine that we have many applications that serve different purposes for the communities. We also have a large and sophisticated operational environment.  The systems team builds the eBay OS, which comprises security, kernal layers, management, infrastructure, and logging.  The entire reusable software infrastructure makes it easier and faster to develop applications.

EL: How do you align with IT?

JB: We're part of eBay's product development organization. eBay differs from most e-commerce organizations because the software we create becomes the business. We align in multiple respects with the strategic roadmaps. We, however, run a separate technology strategy, which aligns with both the business and the needs of the product development organization, as well as the needs of the operations teams. Every year we bubble up a few major initiatives that become the strategic enhancements we need.

EL: Do you have a formal process for  driving sustainable innovation?

JB: Absolutely! We've done that across several different dimensions. On one hand, we have very far-reaching breakthrough types of innovation. These things come out of our eBay research lab. We have a separate bucket of technology investments, which we call our headroom process. It's a combination of making investments in new technologies, but it's also about going through and improving our infrastructure where we know we're going to need scaling. We've done that process to drive up the eBay infrastructure. Every year we have a certain amount for our software development budget. We carve out funds for infrastructure investment, which we then use to innovate on our platform.

EL: Can you describe your architecture platform, which provides SOA components, security management, as well as other technologies?

JB: Those are a few big areas. We've created a very nice, zone infrastructure. It's a full application and architecture for making it easy to create services within services. It also makes it possible to plug things in, so we aren't wedded to one particular appliance or protocol. That's  going to be a major enabler of the business going forward.

We have  many dimensions to security management, everything from writing the professional rosters to locking the front doors. eBay, being one of the largest Web sites in the world, gets targeted constantly. Over the years, we've hardened our system to be able to handle everything that has come at us. As for analytics, we've one of the world's leading data marts and analytics infrastructure. It enables us to process data, to have insight into actions, and to be able to drive business strategies from the data we're saving

EL: Any other technologies that are part of the  architecture?

JB: We use a lot of Solaris. We built our own search subsystem. We use Java very heavily. We built a whole Java framework on top of hung Java. We work with IBM. We have some Cisco in the shop. We have a lot of enterprise tools. Where there are limits to those tools, we've built up additional layers of infrastructure that really solve many of the problems we have at our scale. Two of our other vendors include Oracle and Hitachi.

EL: Can you describe how your group works with  third-party developers?

JB: That program is in my systems group. With the new Web generation, we want to strike the right balance between being open and encouraging entrepreneurship, while having oversight to ensure quality. For example, we have an online directory of different applications people have built. We allow the open community to rate those applications. We then raise visibility or lower visibility based on feedback. It's an inherent built-in mechanism for those third-party developers to create quality.  It's very much like any community-driven process where we try to get away from business-oriented regulation and, instead, to embrace the community to both reward great applications. However, we also make sure we limit our quality products. We find it's a better and a more scalable mechanism for an environment like ours, which is in constant flux. It's a better oversight method than any other rigid process that we might have invented.

EL: Do you use  any quality practices such as Six Sigma or COBIT?

JB: We've integrated the best parts of those two methodologies and created an eBay product development lifecycle (PDLC). We have a very structured process where we have requirements, scopes, projects, and everything for QA. Because we built this eBay OS, we have invested heavily in automating our entire development and rollout process. We can easily manage 100s and 100s of developers, and 100s and 100s of projects going through the same PDLC. We can allow a small one- or two-person development team to go through the same process and tools as a 50- person development team. It scales. We've tried to take a lot of the redundant human element out of it. As a result, we really enforce those best practices and those standards around development and quality, and management of our release process, as well.

EL: How often do you release your  OS?

JB: We roll out our entire site every two weeks. One week we roll out our North American site, and the following week, we roll out all the international sites.  We constantly re-roll our entire stack. Every single one of those releases has features, additional functions, and new applications going out.

EL: Because you're mostly a build versus a by shop, what advice what you would give to IT shops steeped in integrating custom products with third-party solutions?

JB: I'd tell them to be careful not to be a fast food order taker and just to build what you hear the business asking for. When you're building custom software, you're making a big decision and a very significant investment in this process. It has to be long lived. Don't deliver what you hear. You need to be a partner and to craft solutions that meet the heart of what the business is after. On the other hand, you have to balance the costs of custom development to make sure you're hitting that core strategic need of the business. You also have to craft that solution with them and strike the right balance.

EL: While eBay continues to scale,  how do you remain agile?

JB: We've been investing in a program called Nexus. It's an infrastructure that has a lightweight, satellite component. This eBox allows a very small team to be very rapid and nimble. Likewise, it's the same software stack and infrastructure that we can build an application that is serving 100 million users. This vehicle enables a small innovation team to crank out something, such as a marketing application, very quickly. If you do a 100 of these, then a few of them are going to be wildly successful. You're going to need to scale them. Rather than rewriting them, we're putting them on an infrastructure, or the eBay platform, which is going to allow us to scale to an enterprise class, 24 by 7 highly available system.  We've completed a few releases of this. A team of PHP developers Germany came in fresh to our technologies. They had rapidly learned ebox, We developed an application faster than we could previously. It was very successful. That's how we've extended our infrastructure for rapid innovation and very mature high quality applications as well.

EL: What kind of a development  program do you have for your staff to continue to innovate?

JB: We do a few things. A lot of our projects involve trying to solve problems related to our scale. It's a job of making it better.  We have people who find  fixes to low-level operating system issues and JBM's. They're contributing their fixes back to the open source community, as well as pushing vendors to improve their solutions.

I can't tell you how many times our junior-level engineers find solutions to a major vendor's product. On the job, you have to be an innovator. You have to continue learning and extending your skills. eBay also has a development program for tuition reimbursement for education. We also run internal training programs.

We run all kinds of innovation programs. We call them Innovation Days and Skunkworks programs. I'm on the scoring committee for the next one we're going to do. We allow people to create any kind of idea or wildly creative solution they might've thought of. Typically, they build it on the eBay APIs. We reward and promote those different applications. One of these applications went live on the site. For example, a product, called eBay Countdown, came out of Skunkworks.  It's a Web 2.0 Flash-based application that let's our community watch items as the auction counts down. That's an example that went from concept to live on the site quickly. eBay To Go, which is another one of our most popular widgets, is another example of an innovation.

EL:  What are you doing to help large sellers become  more efficient?

JB: We focus constantly on seller efficiency. If you think about the entire developer's program, we have more than 55 percent of our eBay listings coming through the APIs. We've built an entire set of software, infrastructure, and APIs to allow sellers to automate their businesses. You can see just by the volume over the years. The sellers are more efficient.

EL: How transparent is your technology to your  constituents?

JB: We're very transparent in what we do. We're clear on the levels of the investments we made in business projects, in our infrastructure, and our research and development lab.  Even within those infrastructure investments, we have transparency about what we're doing. It has to be aligned with what the business wants.  The Nexus program drives both innovation and flexibility. It as much a technology need as it is a business need.  How do we innovate faster? How do we have shorter time to market? How do we lower costs? Any business person is going to be excited about any solution that can do these things.

EL: Have you gone to server  virtualization or grid computing?

JB: We are doing a lot in virtualization with a few different virtualization technologies.  Some parts of our system are running in a virtualized environment.  Our quality assurance has seen dramatic benefits from virtualization. Not all of our product environments will benefit from virtualization.

We've invested heavily in grid computing. For example, we've automated our entire release process. When a software developer checks in his or her code, an entire workflow gets merged to the main system, and then goes through regression testing on QA. We have an automated environment that rolls it out and distributes it to our site. On top of that, we have all types of analytics and problem detection. We can see from release to release if there is a dramatic variation in how long a process is taking. That will flag problems for us. A number of automated tools help us to triage and to influence what the problem is.

Author: Elizabeth M. Ferrarini -- She is a technology writer  from Boston, Massachusetts. Reach her at elizabethferrarini@yahoo.com.