The Great Job Hunt

Ahh the good ole engineering log. More like my personal organizer and to-do list.

In preparation for the upcoming job interviews I thought it best to look over the following basic technologies just to ensure that I have answers to various questions, and to go over the things that I "supposedly" learned in 413.

Review List:
Servlets
Apache Tomcat
Apache Ant
Junit/Unit Testing
Rest API (via the Hackystat RestApiSpecification: http://code.google.com/p/hackystat-sensorbase-uh/wiki/RestApiSpecification)
Apache Struts
Ruby on Rails

Seems like a reasonable list to review for the present.

Zombie State

Yes well at least I got the demo working a little while afterwards the meeting. Didn't go as well as it should have, but I'm pretty close to being finished, should be completely done sometime next week. Hopefully.

Wanted to blog about the help I got from Hong Bing before I forget what he told me. Basically the data that the jstl buttons receive come from within the form declaration on the jsp page. If there is no data declared between the form tags then the button and subsequent java class will have no data to handle. This is leading to a single button implementation. Also of note is that the user emails will need some kind of remove functionality. There are still some bugs with the xml persistence mechanism and I want to be sure that its happening correctly before I try to break something else.

Sleep Deprivation and H4's

Last night was the first time I slept in almost 4 days.

It's probably a good thing that I hadn't blogged during the interval since it would've been incoherent rambling (much like how I normally blog I guess).

Monday was the first H4 I had done and was with Hong Bing working on my TimerTask implementation. The TimerTask makes a call to its own generateEmailIfTriggered method, making a reference to whether the Alert associated with it is enabled or not. The unfortunate result of my fooling with the TimerTask was general email spam being sent out to myself, Philip, and possibly Aaron and Austen although they said that they were unaffected. Last night even though I had disabled all alerts for every registered user the server was still attempting to send emails to Philip. Good thing that Road Runner is registered on the RBL List as an untrusted source of spam. So I can't send emails to test the module from home, but I can see where it is attempting to send emails to. If you received any spam from me, I apologize.

The hardest thing I'm having to deal with now is the transfer of data from the CommandRequest classes, as any data entered into them seems to get wiped out every time the page gets reloaded. The solution to this I'm assuming is JDOM persistence which is the last thing I need to get done.

Today's H4 was with Robert Brewer and dealt with going through the Agile Web Development with Rails book and the tutorials found therein. It highlighted the Ruby's interfacing to a database, creating and validating data and formatting the data for display on the web. I really hope I have some time in the near future to begin programming Rails although the sense that so many things are hidden in the implementations is slightly unnerving. I think its the same with any new programming language you encounter since Java often does the same thing.

This friday is the final deadline for the project and I'll be visiting Referentia in the morning. I still need to write up a user and administrator guide tonight. So its time to make the last burst for the finish line.

Tutorials bonanza

Some of us are graduating soon so I thought I'd put some stuff together for us to keep out collective hacking skills together. For the past year or so I've been amassing a collection of tutorials from digg, of which I have actually gone through very few. Some of them are general computer science knowledge, but most do represent technologies which we have touched upon this semester and I thought I should share them in order to provide those of us who will soon be having a lot of free time something to keep our technological chops in good condition.

Top 12 Ruby on Rails tutorials


5 books every programmer should read

The Ajax toolkit framework for Eclipse

Over 140 Ajax tutorials

Python Tutorials (only because I know absolutely nothing about Python)

Computer Science Cheat Sheet

54 general programming exercises

Keep those coding skills up to snuff graduating class no matter what you're doing from here on out.

Echo2 and the REST Architecture

I was really impressed with the REST architecture as outlined by Phillip today. Especially with the ease of the way in which modules interact via HTTP get, put, etc commands. Instead of the big black box program that it was before splitting up the system into smaller black boxes which communicate via such simple procedures seems infinitely easier than the hidden communication channels that you had to hunt down before. When I first started out developing for Hackystat I felt like it was this big black doorless house that I was stumbling around in the dark looking for a way in. Eventually some of the hidden doors revealed themselves through the wood paneling, or I jumped through some windows, but under the rest architecture its only a bunch of outhouse shaped buildings that have clearly defined paths between them. From a developer standpoint this is infinitely better than how it was before.

I'm have a probable entry point for my TimerTask to intrude upon the Hackystat architecture, and once its working will finish the alert portion of the module. I still need to find a way to set a different time interval at which the emails get triggered for testing purposes though.

Working on the UI as I write this.

Peer UI Programming

The data gathering bug, which resulted in me gathering data for only my single user instance instead of gathering data for every user and consolidating them in a single daily project data instance, was a result of me not being logged in as the administrator of the project over which I was trying to gather data.

When logged in as the owner of a project the mail and all its formatting seems to work fine, although I will need more data in order to verify this. Right now have switched all efforts onto working on the alert implementation of the emailer which includes its own timer task implementation because the default Alert interface was dependent upon the default timer task, and therefore the default emailer which does not support html emails. Speaking of which I still need to figure out why the email does not display properly when viewed in gmail. As Robert says it is probably the external stylesheet and .gif links embedded. I will have to find a way to circumvent this eventually. With the TimerTask I need to figure out: 1) How to implement a second differential instead of a single set hour each day, 2) Where the alert makes calls to the actual Java code.

So the final to do list for the remainder of the semester looks like this:
1) Alert
2) Resolve Email issues
3) Fix the UI data gathering issues
4) Persist the data gathered from the UI

Wow I might actually finish this thing on time.

I have no idea what I'm going to talk about tommorrow.

Yay.

Houses that Open and Ghosts in the Shell

The open house event was fun and informative since up until now I've never been privy to any of the products and services that they provide to the DoD. Models and Simulation, Autonomous Agent systems and Data Mining are all things that I love reading about and hope I have a chance to work on at some point later, if not for a company then certainly from a personal research perspective.

An autonomous agent system for Hackystat would be interesting. Sending the agents out into the web, to crawl and collect data, which they send back to the central Hackystat repository. Would be cool to see in action. I especially liked the microwave emitter demo and their dispersion algorithm. I still would like to do something modeling systems of objects in their relationship to one another. Like the swarming or flocking presentations I've given in other classes.

Would any of it work for Hackystat, who knows. I haven't even taken an AI course yet =\. Remind me to ask the guys in the lab for a good AI book.

Still still still trying to track down the dpd bug. I think it might have something to do with the fact that sending a single project instance to the data gatherer returns data, but iterating over all the projects owned by a user returns the indication that every project has zero data. Hmmm. The interface I don't think is a bug rather than I need to implement the request hooks instead of just trying to pull the data from the main UI java class. Should shoot for getting this done by this weekend.

Golgotha Tenement Blues

Finally finally finally finished the html template after reducing the code to almost a quarter of what it was originally. I was making it much harder than it actually was and as a result of trying to gather too much metadata regarding the daily project data I revealed how little I actually knew about how the data was structured. Once I actually set out to see how the data was actually structured the answer to my formatting problems fell out immediately. Took way too much time figuring this out so now I really need to bust ass with regards to the UI. Hopefully I'll be able to figure out the data gathering problems quickly.

HTML Templating and Daily Project Data Collection

There seems to be an issue in the gathering of the daily project data with regards to gathering data for all user for a given day, as opposed to just data for the user logged in. Right now the data is gathered via the DailyProjectDataCache, which I'm thinking might not be the right implementation for this project. When the Daily Project Data instance is returned from the cache the code calls the getDrilldowns method, which returns the drilldowns for the indicated day.

What I don't understand is why the getDrilldowns method uses the 'User' parameter. Right now I'm only able to collect data for the user logged in, but when the DailyProjectDetails module uses this method and invokes it in the same way through the data cache, it is able to grab data for all users.

I've dug through almost every Sdt extension of the Daily Project Data abstract class and can't see ANYWHERE that the user parameter is used.

If anyone knows the answer please let me know.

I should be finished templating the HTML code tonight but I need all of the data from all users to populate it, or else the whole purpose of this project is meaningless.

Structure Review and Project Status of the ProjectStatus Module

I can say with confidence that the back end of the program is basically complete, with the exception of a few bits and pieces here and there. Most of the remaining back end has to do with the alert implementation.

What now remains to be done has almost primarily to do with the UI. The data flow of the UI and exception throwing pages. The alert enabling and the email entry. Persisting the email addresses entered and checking their validity also needs to be addressed. This persisting of the emails via JDOM and verifying their validity is probably the largest portion of the project that still needs to be addressed.

With the completion of the Java1.5 conversion complete i'll have more time to concentrate on these issues.

Class Cast Exceptions and Java 1.5

Well i'm slightly happy to note that all of the frustration earlier today was due to the Java 1.5 conversion. The whole time I had been getting a sneaky ClassCastException which didn't show itself until I put some output code directly into the DailyProjectDetails class. The idea was that the getDailyProjectDrilldowns method is supposed to return a List of a List of String[] (blog won't let me post angle brackets due to html rejection). However it turns out that this was only happening for MOST of the SDT's, with some of them returning god knows what. When the code would hit these SDT's which did not return the expected Types the code would bomb without any indication why.

Thus all efforts are now turned to the Java 1.5 conversion (which I should have been spending more time on anyway). After all it turns out that the SDT's the code was bombing out on were the ones i'm responsible for... =-(

Drilldown Issues

I wish subversion had a browser that cached all of your old versions, so that you could browse through your old code in case you changed something that you needed but didn't necessarily want to revert everything back to that previous version.

I didn't have a problem early getting the daily project details summary strings, but for some reason i'm having a much harder time getting the drilldowns. I can't tell if i'm trying to look at the wrong day's data, or if there is no data associated with this user for this particular day. Or if i'm just messing up on the data gathering side. Or maybe the project isn't getting loaded correctly for that particular user when hackystat gets deployed.

Now i'm just venting. I need to keep track of these issues and eliminate them one by one. If I could even determine that there is indeed data to look at before I try to manipulate it.

Use Cases and Data Compartmentalization

Meant to blog yesterday but fell asleep trying to think of what to write.

I refactored the package structure to resemble an MVC design architecture since I was having trouble separating the view from the data collection. I wanted to have the View able to call the sendEmail method while passing the information it had gathered from the UI (such as email addresses) without needing to touch the data at all. I refactored and moved files around to reflect the data flow more accurately, only to discover after doing so that this is not MVC after all.

At the moment all the view does is gather data from the UI and call some sendEmail method in the mailer. The mailer gathers this data from the view and calls some getEmailBody method in the data manager which returns a String representing the email body. What the mailer doesn't need to know is that this String getting returned is completely made up of html tags. In essence the mailer shouldn't have to touch the data at all either.

All of the data collection and formatting should only be occuring in the data manager. The only thing the data manager may need is the User name, in order to get the names of the projects owned by the user. These projects with their associated data, and the html template page, are the only things the data manager should require (I think). It should take care of all formatting and only need to return a String, which the mailer cannot discern from any ordinary emailBody String.

I think this method of comparmentalizing the data should work out well.

Oh yes and I really need to write a detailed use case for this new module design in order to eventually finish the UI.

Pair Programming and Summary Grids

The Pair Programming last night was extremely helpful. Much thanks again to Austen. Some of the items we discussed included:

1) HTML tag replacement and recognition. To dynamically generate the HTML page the module needs to recognize the specified HTML tags and replace them with the title followed by the data from the drill down lists.
2) The difference between how the summary strings are formed as opposed to the drill down data.
3) The typing of the List returned from the summary grid containing the drill down data and how to access it.
4) Issues regarding whether every sdt implements its drill down data uniquely.

Some miscellaneous things we went over included unit testing, proper java styles, and especially shortcuts and hotkeying things in eclipse. Which is just the greatest thing ever quite frankly.

This accessing and instantiating the drill down data remains the hardest thing left to do in the project. The remaining items to finish include:
1) Finish the UI/fix the UI bug(s).
2) Finish the new timer task implementation.
3) Figure out why gmail doesn't like linking to external style sheets.
4) Persist the entered email addresses and boolean "enable" value using JDOM.
5) I also need to update just about everything on the google project website.

At the very least I need to have at least a manual email generator which displays the drill down data in a formatted email finished by next week. I'm not sure what to package for the release this friday though.

3/3 - UI Work

Trudging through the UI, learning more and more jsp tags and incorporation of java code into the jsp pages in order to dynamically name the variables which the command will be pulling data from. Ultimately i'm moving in the direction of working on Persisting the data with JDOM which i'll hopefully get to today.

Going proves slow though.

Mood: Resilient

3/30 - To Do List

This list is the result of talks this week between myself Phillip Johnson, Austen and Aaron

1) Stream Daily Project Data (does it need to be cached?) and read it into the module.

2) Implement Timer Task (use 30 second intervals for development

3) Redo UI, new JSP pages closer to the project management UI

4) Persist emails entered into UI textfields using JDOM

5) Create unit tests

3/29 - Forks in the road

This new direction of the project begs the question of what to do with the old code and whether it should still constitute the presentation to the class in terms of release packages.

Must be sure to clarify during the meeting tommorrow whether they still want this old functionality or if it should be scrapped.

3/29

Points of interest from the Peer Programming session with Dr. Johnson:

1) Remember to use JDOM to persist the attributes and data defined by the users within the module.
2) The only capability that needs to be examined it probably the TimerTask which I haven't touched thus far.
3) First thing to do is probably to examine the structure of the ProjectManager to get an overview of how the user interface should be constructed. This is probably the best place to start.

...go

Go

"The Ancient Japanese considered the Go board to be a microcosm of the Universe. Although when it is empty it appears to be simple and ordered, in fact, the possibilities of gameplay are endless. They say that no two go games have ever been alike. Just like snowflakes. So the Go board actually represents an extremely complex and chaotic universe."

- Sol Robinson from the movie Pi

2/27

Didn't realize that I haven't been posting in a while.

Things finished:
1) Finally got the interval jsp page written and have successfully retrieved the interval selector from the jsp page. Now just need to figure out what to do with it, ie. how to implement only daily and/or weekly alert triggers. Doesn't really matter though because the weekly analysis is something else entirely.
2) Also successfully sent an html email, although the formatting is fucked because of the List.toString() method. Should be able to find some other way around this. concat(String)?

Things to do/goals for 2.0:
1) Need to figure out why the multipart email is throwing a MessagingException, only able to send the html email, and not the text/html emails both simultaneously.
2) Still need to port the system over to an alert instead of manual invocation. Possible topic for the peer programming but I should be able to do this one on my own.
3) Of especial interest, possibly a topic for the peer programming, need to find a way to cache the daily project detail html output and how to retrieve this html page to send it as an email.

These goals should comprise the v2.0 release for next week.

3/20

At Referentia right now.

Next Phase will be the idea we discussed before about combining the different telemetry streams in order to consolidate the different chart reports into either 1 or 2 charts at the most.

Necessary things to accomplish:

1) Finish reading Cedric's thesis.
2) Decipher the telemetry stream language.
3) Complete the Alert module for next week's release.

Considering trying to make this into an AJAX implementation is this would prove even possible. Should propose this idea to Phillip.

Issues:

1) Lots of things are hard coded into the module at the moment. Must spend some time refactoring these before the release.
2) Considering just starting the project over from scratch, copying the core module code into a new alert module and keeping the existing code as a testing suite for manual testing. Reasons: converting the module to an alert while still retaining testing functionality is proving problematic.

3/19

To Do List:

1) Get HTML into the emails.

2) Finish Refactoring the Module.

3) Write the XSL script and subsequent HTML --> email loading.

2/16

Didn't get a chance to comment on the CSDL meeting wednesday.

Of especial interest was the idea of dynamic graphs, graphical data which although being displayed graphically in charts or graphs could also contain drill down abilities. Such as the pie charts, which initially represent project level statistics but when clicked upon separate into separate pie charts which represent package or module level statistics.

And yes the comet tail idea wherein a spectrum of colors in the shape of a comet tail could be representative of project level metrics.

3/12

Things currently trying to get accomplished by the end of the week.

1) Finish HTML formatting of daily project details.

2) Finish JSP page design for the interval selector class.

3) Finish alert class with respect to the getDisplayUrl method.

Will add more if i can think of anything else. Need to go into the lab tommorrow to ask questions to Hong Bing and Phillip. Also need to think of some analysis questions for the CSDL presentation on wednesday.

3/10

Starting work again after being sidelined by disease.

3/6

Went to Referentia again around 11am.

Realized that I still have both the alert class to finish writing as well as getting the interval selector jsp page to work. Hoping to finish both of these today and start on the html page design tommorrow. Hopefully dreamweaver will be able to help come up with some design flow ideas. Possibly something to talk about in next week's CSDL meeting or in the lab.

Be sure to annotate the developer guides and use cases with screenshots before friday.

Trying to get back on track with the programming schedule. As the size of the program grows there's more and more stuff to tweak and bugs to fix.

Really really need to take some time to work on unit testing this thing.

3/5

At referentia right now, working on hardcoding the data into an html file as a prototype to the dynamic html generation which will finalize the format of the emails.

3/1

Collaboration with Referentia went well today. DailyProjectData collection is working but only in string format. Still need to port the main ProjectStatus implementation over to an Alert implementation instead of manual invocation. Also need to create the interval selector class and its associated jsp.

Must make time either monday or tuesday to go back to Referentia to do more collaboration.

2/27

Spending some time figuring out if the alert should be project specific and how much harder that would be.

At the moment it would seem that a map dataset would need to be kept somewhere. Where the projectNames would represent the keys, and the values would be represented by boolean enabled/disabled values. Not sure if they would want this functionality but we'll try implementing it before thursday.

Maybe getting somewhat off track here, should probably go back to trying to collect the daily project details tommorrow and figuring out why that's not working first before implementing any new functionality.

2/25

Currently working on completing 3 things before the next Referentia meeting this coming week.

1) Must retrieve the dailyProjectDetails data from the stdCmd alert class. Might not be able to use this method to retrieve the data.

2) Need to write the ProjectSelector class to select which project the data will represent.

3) Need to recreate the drop down list on the jsp page and populate it with projects.

2/23 Version 1.0 release class

Sitting in class right now.

Issues:
1) Annotate the hackyStat statistics screenshots.

2) Add screenshots to the guides and use cases.

Demo's went well, need to do 2 things for the next week. Populate the body of the emails with data from dailyProjectDetails, and implement a timeframe selector for the jsp page.

Larger goals for the next week include html generation of data via xslt scripts to populate the body of the emails.

2/22

Worked all night on overhauling the project from its old model to the newer integrated design which actually plugs into the hackystat system instead of standing on its own. Still stuck on the Apache bug documented on the news page however.

Be sure to ask Austen and Aaron about the concerns brought up at the CSDL meeting, as well as the idea that this email functionality already exists in another form within the hackyStat system.

Be sure to ask Phillip about the bug.

2/21

Some future things of note to ask during next Referentia meeting.

What are the goals they wish to achieve with the new email functionality? Will they want to extend the alert system to other aspects of either the developer process or events within the projects they are working on.

2/19-2/20

Make extensive revisions to the system, added a new package to the system which will house the Alert section of the program. Running into some workspace issues when trying to see data, or possibly some sensor issues when trying to send data, not sure which it is. Currently working on powerpoint presentation to give to CSDL tommorrow. Really not too sure what i'm going to say. I'll try reading Cedric's paper for some ideas.

2/15

Got my first sense of agile programming today. Had a meeting with the Referentia staff today at 12:30. Upon review of the previous requirements by Aaron, Ryan and Austen they decided to take the project in a different direction for the time being.

Instead they now want me to concentrate on developing a branch of the current application module which can autogenerate emails on a to-be-determined frequency. The content of the emails is an alteration of the DailyProjectDetails and the recipient of the email will be the developer to whom the data pertains.

Basically they want to be able to report back statistics to individual developers on all the normal data collected. This is combining statistics reporting with a reformatting of the DailyProjectDetails.

For the past week I had been working on the MVC design for the managment reporting but Aaron says to put a hold on that for the time being.

I told them that's fine and that i'll start working on this new aspect of the project instead.

2/10-2/13

For the past 4 days i've been working on updating the content of the google projects webpage so that it covers what i am doing as well as what i hope to accomplish extensively.

My primary goal at this point is to go back over the hackyStat documentation to gain further understanding of the hackystat system, and to find whether the implementations which i hope to incorporate into my application exist already. I'm just hoping at this point not to have to build the data collection and analysis portion extensively from scratch.

2/9

Entered in the first use case and the issues hoping to accomplish by the 1.0 release.

2/8

2/8 – Meeting with Referentia

Aaron wants reporting for: code issues, file metrics, coverage, commits.

Said to look at http://tsubaki/hackystat/controller?Key=S5ASQkXWuzPK&Page=analyses as a reference.

Create a confluence page.

Find a suitable App to model the ProjectStatus app after.

Ask Philip if I should be using telemetry for the trend lines or not.

First Post

HAH!!