All Things Software

Hello, its sure been a while since I have had a chance to blog about anything.  This summer sure has gone fast, with migration after another.  Personally, I'm sure glad the semester has started, so I can get caught up on other things.  Today's topic is maintenance programming, that is art of developing software to maintain existing applications and/or services.  At my previous job at Raytheon, that used to be the kiss of death.  Comments like, "Oh he is doing maintenance, well that's it for him" used to be commonplace.  Reality is, I have a friend who is doing maintenance on the same software application that I worked on 12 years ago.  My friend knows its a real killer, but he has all of the experience so his skill is needed.

As of the last two years, I am finding myself in the same boat, just doing maintenance programming.  Needless to say, I know how important it is, but in a word it can be just boring.  How do you make it exciting, other than the obivious resigning?  Well what I try to do is see if I can apply a new technology or programming language in a way that will enhance the functionality of the application.  Of course you do not want to do technology for technology's sake. 

Here is an example of where some new technology could make things better.  We have some applications that provide APIs that when called returned XML documents.  Back in the day, when I developed them, frameworks, SOAP, Web Services and so on were just in their infancy.  So doing things with a simple HTTPS POST and rolling your XML was pretty much the solution.  Now today, we have things like Web Services, and REST.  So for some of the newer enhancements, I will be looking at those technologies.  Hopefully that will make maintanance programming more exciting.

I'm not sure I will keeping this blog around or not.  That's another thing I have been toying with too.   

You are probably asking yourself, which Portal?  There are so many things out there today with portal-like features, that's it hard to tell one from the other.  The Portal that I am referring to, is the Penn State Portal @ https://portal.psu.edu/.  As many of you know a long time ago, I was the primary developer for the Portal.  I'm not going into all of the historical stuff, I already did that in a previous post. 

When we looked at the portal concept, a number of us felt it was great idea.  One place to go to for all of your stuff, whether its Penn State-related or external like CNN.com.  And for a period of time, we had some traction.  The portal was the place to go for a number of things like RSS feeds and chat.  When we were doing our design, we make a conscious decision that we were going to settle on using RSS for content-type channels.  At the time the specification was in the hands of Netscape and the version was around 0.91.  Software to take RSS and convert it to HTML existed, but its performance was not very good.  So we ended up writing our own parsers using compiler tools (Lex and Yacc).  Then for chat, we had a Java Applet (yes an Applet) that supported Jabber in the portal.  Another thing that we can up with in the portal was the concept of your personal bookmarks.  That way a person could log in and always have access to their bookmarks.  Again another great idea at the time, now with sites like del.icio.us, you may not need a capability like that.

So where have things gone?  Well today, everybody and their brother has their own RSS aggregrator (News Reader).  I personally use NetNewsWire.  And for things like chat, we just launched a Jabber Alpha/Beta that supports native clients.  A web-based version is on the way.  So there are  a number of things out there that replace the functionality of the Portal.  I will be the first to admit that. I have not checked usage numbers in years.  But, people still use the portal.  Why you may ask?  Well it gives them something that many other sites do not, personalization and customization.  Your portal is personalized for you, because we know a little bit about you based on your directory information.  This is a pretty neat concept that still has merit.  Customization of course will allow you to go in and pick and choose the concept you want to see. 

Back to my original question, is the Portal dead?  I think it is on life support, but not dead.  People at Penn State and other Universities still find value in the portal concept.  Remember a single place to go for all of my stuff.  If I had my way, what would I do to resurrect the portal?

1. Complete rewrite to make it Web 2.0/3.0.
2. Make content publishing more user friendly.
3. Try to make it the hub for all Penn State applications.  Those applications do not have to be wholly contained in the portal, just use the portal as a launching point for them.
4. Make things easier to use (look at Google's Portal for example).  And you maybe asking why not just use Google?  That question has been asked in different forms before, because Penn State content should stay inside of Penn State.

Since others and myself have spent so much time on the portal, I sure hope we can save it and breathe new life into it.

Types of Code Story (Waterfall):

Back in the day, when I was at Raytheon I was involved with writing a number of proposals for a variety of things.  Sometimes the work was interesting other times it was real boring.  One part that was interesting were the line of code estimates.  For a paragraph of text that describe a particular function, you would have to come up with a line of code estimate for it along with the percentage breakdown based on type.  Then the financial folks would take that number and determine a cost.  Sometimes the estimate would give us a win, others times not so much as my friend Mark Miller would say.

The types of code that we would use had three categories: hard, medium and easy.  The productivity of each one was 10 LOC/day, 13 LOC/day and 15 LOC/day respectively.  So you are probably wondering what the heck?  A person is only going to develop 10 lines of code in a single day if the code was deemed hard.  Well, in a word yes.  Those numbers included more than just writing 10 lines of code.  It included all of the up-front design, reviews, coding and testing.  So in the end you were getting fully tested code.  Back then we were a waterfall shop and now from what I hear things have more on into other methodologies.  Here are some examples of each of the types of code:

• Hard - device driver development.  Yes, back then we did that since we built and engineered our own hardware.
• Medium - interface development.  Writing X11 programs at the time was not very easy.
• Easy - simple utilities.

WebMail (Agile):
Now in present day, we are doing some great strides with Agile techniques for WebMail.  Our development cycles are typically 2-3 weeks.  Within that time, the planned features are discussed, code is developed and full regression testing is performed.  I am very pleased with the product that our team has put together.  On May 12th, we will be doing another launch of WebMail.  This one which has been in beta since Spring Break, will enable us to turn off WebMail2.  The new version of WebMail will have all of the existing functionality of WebMail2 and loads more.

My final story (SCRUM):
And finally my story, one of the tenants of the Agile project management process known as SCRUM has to do a lot with how the team functions and communicates.  We (Applied Information Technologies, Emerging Technologies and others) participated in a SCRUM effort a while ago and probably none of us knew it.  The task was to take Shibboleth, and use that as the basis of authenticating with Napster.  Shibboleth enabled our Penn State students to authenticate at Web site and have only that they were a student passed onto Napster.  Napster never knew who the student was, just that they were a student. There are many parts of shib and I really do not want to go into them this morning.  Back to the SCRUM portion of the post, we (AIT) never worked with Shibboleth, heck for the longest time, I could not even spell it.  ET had a pilot already going, but now we had to ramp up for large scale production and we had less than 30 days to do it.  There were so many unknowns with performance, like how many machines do you need?  At the time, Kevin Morooney was the Sr. Director for ASET, he of course realized how important the project was, like all of us did.  However he did something novel which is part of SCRUM.  We would meet for 10-20 minutes every morning go over the day's objectives and previous day's results.  Then at the end of the day we would meet again to see how we did and plan the next day's objectives.  Let me tell you, this method worked out great.  How often do you find yourself in a situation where you have a major effort going on and you are only meeting weekly, bi-monthly or even worse monthly?  If the effort is critical, you need to take a page out of SCRUM's play book and do the daily meetings.  It worked out great for us.  After all of the planning and testing we did, when it came time for that first student to log in, we were watching the machines and things worked.

So there you have it, the past, present and the future all wrapped up in three little stories.   

What a title?  RISK (even in uppercase), pretty scary isn't it?  I did a quick google for some definitions and I really like this one:

Risk: The uncertainty of an event occurring that could have an impact on the achievement of objectives. Risk is measured in terms of consequences and likelihood.

By the way, I am not talking about the game of Risk, which was the first hit from google.  No, folks I talking about risk with regards to a project.  All items of risk should be taken into account during all phases of a project.  In addition, the affects of risk should be accounted for in the overall schedule.  Here are a couple of examples of risk that can affect a project:

• Dealing with an vendor that delivers hardware that is either missing or not functioning.
• What do you do if you technical lead or other key player decides to resign or worse becomes ill (the hit by a bus scenario).

And you can look on-line to find many more examples of risk.  When I was working at Raytheon all of the proposals that we wrote had a special section in them to list the risks associated with the project.  For each risk we also had to list how we would mitigate the risk.  This of course affected our bottom line, and being a profit driven organization it was all about making money.

So what do you do about risk?  Well first off, you need to acknowledge that it exists.  You would be a fool not to.  Look at the all of the external forces that can affect your project.  Each force represents a risk, whether its a hardware vendor or the company doing your copies.  You need to acknowledge the risk and come up for a plan to mitigate said risk.  The same thing applies for internal forces too.  Risk exist there too, things like staffing and commitment all affect the overall project plan.  Risks, like everything associated with a project needs to be constantly evaluated, because at different stages of the project the level of risk could change.  Early on in the project, the technical lead leaving would have less risk than later during the overall development. 

I used to be accused of padding estimates when I gave them.  Why?  Well I knew some of the risks, and wanted to take them into account.  Reality is these days, I still do that.  Because worrying about risk and its affects, brings me to my final point.  Be Realistic!  Do not kid yourself that things will go perfectly, they never do.  However by acknowledging risk and its associated affects can make things easier when things just do not go right...

Communications can probably be best described as a tower of babel.  People are there and they hear you speaking, but what the understand and hear is completely different from the person sitting next to them.  Today's posting is going to be about communications as you have already guessed, not about some fancy software technology.  Communication in general is an integral part of any process including a software one.  So it is very relevant to the theme of my blog.

So how do we effectively communicate a new process to a group?  Let's look at a way not to do it first and then some suggestions how to effectively communicate.  As many of you know, one of my duties is being responsible for the architecture, programming and care of Penn State's LDAP servers.  I have been doing this for a number of years now.  Because of that, I am involved with a number of committees and I have to attend meetings about LDAP.  So its been probably three years now, I was called into a project meeting to discuss LDAP.  The meeting was supposed to be about how LDAP could be utilized for role-based authorization (another area of keen interest to me).  So of course I had many answers and comments.  People were asking questions and I gave them answers in painful detail to include how the protocol worked.  There were times I was having sidebar chats with my boss about how we need to do things in the future.  All of these things and many others did not nothing to facilitate effective communication.  In most cases, I probably alienated the audience.  Why?  Because my communications with them were flawed in many ways.  Trust me, it took a great course called ITLP to help me understand that.

Now for the suggestions on how to do better effective communications:

1. Its all about the message.  You need to frame your message so that the audience can understand it all levels and that they have the proper take-aways.  This is a easy one if you have an idea of who your audience is, but what about if you do not.  Well you start out simple and see where the discussion leads.  Remember you are there to help them solve a problem, always keep that in the back of your mind.
2. Keep the presentations so that only one person presents.  I know that tag-teaming presentations is a great concept, but in most avenues its confusing to the user.  You need a consistent presentation that flows from topic to topic.  Taking time for a new speaker to set up, causes the audiences attention to wander.  Do not believe me, just watch them.
3. Make sure you have a focused agenda for your presentation, and stick to said agenda.  The worse thing you can do is just wing it.  If you know something that your solution that is problematic or will not work, do not take vital time away from your presentation to sit there and debate solutions with your team.  Take the discussion off-line.
4. After you make a presentation to a group, try to follow it up with an email that re-enforces your important points that you just presented.
5. Take aways - In the world we will in, being Green is in.  I know that killing trees is a bad idea.  However in situations like this, giving the audience something they can take back to their office will not only re-enforce the presentation, but hopefully serve as a constant reminder of what happened.
6. When you are giving presentations and communicating with users keep the presentation portions short.  You do not want to dump extra detail on them that they really do not need.  Of course be ready to answer their questions if they have them.  But, make sure you keep those answers short.  Communicate to them a solution to their problem, but be light on the details.

There are many other great resources on-line about how to do effective communications.  I have a number of them bookmarked on my del.icio.us (keyword leadership).

For the longest time now, we have been doing code reviews the old fashioned way.  We would print out the code and two people would sit and review the code line-by-line.  Our notes were either attached to the code or written out and given to the developer beforehand.  This is was pretty much what we did when I was back at Raytheon, except we had more people involved. 

This morning, I came across a really good tool to help facilitate code reviews.  It is called Codestriker.  It is written in Perl and uses MySQL as a back end.  I put a version up on one of my Linux boxes and created a database on the MySQL cluster we have.  The tool is really feature-rich, in that it can work with CVS, SVN and other source code repositories.  I have at least three projects in the queue to review, and most of them are new.  So this morning, I uploaded the code to the tool.  Then I was able to review the code line-by-line.  To add a comment, all I needed to do was click on the line number.  Then when I review the code again, all of the lines that are in "red" have comments associated with them.  The benefits of this tool are:

1. You now have a history of all of the code reviews for a particular project.
2. It enables multiple reviewers to review code at the same time.
3. Developers can log in and address the codes prior to the actual review.
4. Did I mention its open source?

So if you are in the market for a good code review tool, give the striker a try.  I did bookmark on del.icio.us a number of other tools that are commercial products.

No, not with me, how about each other?  Everyone has an AIM account, maybe even a Yahoo one too.  The problem with all of these accounts is your are known by names like starblade2000 (my brother-in-law) or jimmyvPSU (me).  Wouldn't it be nice to use your Penn State Access Account in a secure manner to chat with other Penn Staters?  Well folks, guess what?  We have resurrected our old Jabber server.  Back in the day the Jabber chat only worked by logging into the portal (https://portal.psu.edu/ ).  Now we have a new service (that's in alpha/beta), that is load-balanced, secure and uses Kerberos.  All you need to do is point your favorite Jabber client to chat.psu.edu.  Your jabber id will be your access-id at chat.psu.edu (jvuccolo@chat.psu.edu is mine).  By default we are set up for TLS and SSL, so all of the chats will be secure.  Client-wise there are so many out there to choose from (iChat, Adium, Spark, Fire, Pidgen to name a few).  In addition, because its jabber, you can inter-operate with people who have google talk ids, jabber.org ids and so on.  We also have a conference server up at conferences.chat.psu.edu.  Again, try it out and see what you think... 

For the longest time now, my group has been doing a lot with databases.  We provide databases for classes and research.  In addition, we use it for our people registry (a.k.a. CACTUS).  What the heck Movable Type uses a back-end database.  So we have instances of DB2, Oracle and loads of little MySQL instances installed on separate boxes.  From a maintenance and application development standpoint, this has become a real nightmare.  My lone administrator has to ensure that things are patched in a timely fashion not only across the databases but also the platforms. 

Last year, before we made the decision that things would go forward with MT 4, we needed to solve a fundamental database problem.  The web servers were going to be load balanced, but in the past the instances were not.  So I did some digging around and found two solutions for MySQL.  They were clustering and replication.  Clustering looked very attractive in that it did hot fail-over to other nodes.  However as I dug deeper into the documentation , I came across a lot of deficiencies too.  The software just didn't look like it was ready for prime time.  However replication looked like a really good solution.  We were already using a concept like it with our LDAP servers.  In that you have master nodes and replicas.  If a master fails, you can switch a replica node over to be the new master.  So for MySQL what we have is a master with two replicas.  In the works is probably another master for master-to-master replication.  The nice thing about this little cluster is we can place all of our tiny little databases on one MySQL cluster.  As you can probably guess its really great from a maintenance standpoint.  Performance is really good too.  Right now, we have both MT versions using the cluster along with our Jabber server.  So from a strategic standpoint all our MySQL-like databases will use the cluster.

Now on to the bigger databases.  We have been using Oracle for our people database since 1999.  Oracle has loads of great features that do come at a pretty high cost.  For other applications, we have been using DB2.  We are mainly an IBM shop and DB2 has some attractive features (like built-in Kerberos authentication).  And after version 7.x, it added a stored procedure language.  However that language did not rival Oracle's PL/SQL, and the name was confusing it is called SQL/PL.  Right now we have a our classes databases, Friends of Penn State and some other databases on DB2.  Our strategic plan is to move all of those databases to Oracle.  A wise man said a long time ago that "Oracle gets you by the tools".  In reality that person was correct, their tools far exceed anything that I have seen on DB2 (have you seen APEX?).  Just like with MySQL, the benefits of a single large database like Oracle are really going to outweigh the pain its going to take to migrate some of these databases from DB2.

So where are we at with all of this?  Well from the MySQL side, I would say we are making excellent progress.  We have moved a number of our "baby" databases over to the cluster.  From the large scale side, we are waiting until we migrate to Oracle 10/11 and then we will start the migration of the DB2 databases over to Oracle.

People use that expression a lot when they talk about a time when things were just perfect for them.  There are times when in my IT career, I miss the good old days.  A time very long ago, when my title was Senior Engineer.  Engineer, of what you may ask.  Well Software, of course.  I was at Raytheon and I was engineering new software following tried and true practices and procedures.  Ah, the good old days, such a great time.  I was a Lead Software Engineer (LSWE as we called them), on a number of projects.  That was a good thing too, I had resources available to me, but I did not have to worry about their performance reviews or personal issues.  Unlike what I do here as an IT Manager.  Even though my real title says, Manager, Research Programmer.  You really cannot escape that Manager part, trust me I have tried.

Wait a minute, people are probably thinking being a manager is better, you have people and you can use them to get things done.  Well, folks hate to burst your bubble but it doesn't work that way.  As a manager, you have to deal with the human side of things.  All of the talks, and training courses really cannot prepare you for a lot of these things.  I remember when I became a manager, I was dealing with so many issues my head was spinning.  Then I took courses like Mastering Supervision which only made situations worse because I tried to use some of my new found knowledge.  Some things worked, others failed miserably.  Things that I learned from the IT Leader's Program (ITLP) were actually more useful.  Having a coach like Brian McDonald (MOR Associates) really was another big win for me.  So know you are probably concluding that being a manager now is great.  Well not really...

I truly do miss the good old days.  I am finding that most of the software that I write these days is of the maintenance variety.  Trust me folks that is not really Software Engineering.  About the only new software I have written lately was helping my kid write some Java for her class.  The reality is, I'm not the only one feeling this way.  I have friends outside of Penn State that are still doing maintenance on software they wrote 20 years ago.  I did that maintenance type of work at Raytheon for about a year and then resigned.  I'm not saying that type of work is not important.  Maintenance work keeps machines moving and airplanes flying.  So what I am saying?

Well, I personally missing the days of being a technical lead and having the ability to write new exciting software.  Do I see opportunities for something like that in the future in my current job?  I am not sure, that's the truth.  I do enjoy having the ability to blog about issues like this and I'm guessing there are other "IT Managers" out there that feel the same way.  Will I leave because of this? (dead silence)

I know a good friend of mine, told me I was nuts for going doing this route of IT Manager.  At that time, I really did not believe him, because the glory and the prestige of the title of manager was in front of me.  Now if he would make that comment again, I would probably agree with him.  Oh to yearn for the good 'ole days.