NPM (Node-Package-Manager) Package Install Error

NPM (Node-Package-Manager) Package Install Errors:
Misleading Error Messages

I recently found myself encountering some nearly useless NPM errors when I was trying to install the redis-node package (i.e., Redis.io database connectivity layer for Nodejs), and it did not instantly occur to me why I was seeing error messages during NPM's archive-unpacking operation, especially as the Node-Package-Manager was dumping out a list of errors that made little sense.

Under Linux, NPM install was producing errors that implied I did not have permission to the tmp / temporary directory or directories the package was being unzipped/unpacked into for installation from (i.e., the temporary location the tarball / tar.gz file would be unpacked in).  Under Windows, the same NPM install (under git-bash window) produced different errors that made it appear the downloaded .git package could not be unpacked for god knows what reason, all hidden in a massive error dump that had nothing to do with the REAL reason for the error.

Well, I figured out why the errors occurred (as detailed in this blog entry), and I also learned that this NPM software is a perfect example of what I consider a non-user-friendly interface in that it presents the user with all sorts of completely meaningless error messages in the event of very simple-to-detect issues.  Bottom line: the node package manager is written by geeks, for geeks.  This is not "enterprise grade" software by any means, as it does not have robust error-detection/traps/messages.  In fact, the error-traps it does employ seem to mislead more than assist: this is simply poor design.  (note: I give credit to ANY open-source effort like this though, and I understand why people focus on other functionality vs. "usability").

Node Package Manager Windows Behavior and NPM Error Messages due to using wrong .git file URL

I have been using Node (Nodejs) for "server-side javascript" development on a Windows 7 Pro x64 machine, with Node "installed" as simply the Node.exe download placed in a local directory of c:\node

From within that directory (using the git-bash terminal window on Windows that was installed with the Git SCM tool version 1.7.7.x Windows .exe installer), I executed the following command:

c:\node>node ./npm install -g https://github.com/mranney/node_redis.git

...but, that produces the following error dump. Can you see why? It was all too clear to me a bit later...

c:\node>node ./npm install -g https://github.com/mranney/node_redis.git


npm ERR! couldn't unpack C:\TEMP\npm-1322598499268\1322598499268-0.7676450146827847\tmp.tgz to C:\TEMP\npm-1322598499268\1322598499268-0.7676450146827847\contents
npm ERR! Error: ENOENT, no such file or directory 'C:\TEMP\npm-1322598499268\1322598499268-0.7676450146827847\contents\package\package.json'
npm ERR! Report this *entire* log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>
npm ERR!
npm ERR! System Windows_NT 6.1.7601
npm ERR! command "node" "c:\\node\\npm" "install" "-g" "https://github.com/mranney/node_redis.git"
npm ERR! cwd c:\node
npm ERR! node -v v0.6.3
npm ERR! npm -v 1.0.105
npm ERR! path C:\TEMP\npm-1322598499268\1322598499268-0.7676450146827847\contents\package\package.json
npm ERR! code ENOENT
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     c:\node\npm-debug.log
npm not ok

Notice the URL I mistakenly specified for the install package... THAT is what causes the error.  Yes, something as simple as copying and pasting the incorrect URL from the project's github page will lead to this tragic error-dump.

This is where I went wrong:

...the default URL shown in that textbox on Github (image above) is for the HTTP version of the URL.  I had accidentally copied that and pasted to my command prompt for my "npm install" command, where instead I needed to do this:

Now I could copy and paste the proper git:// prefixed URL address of the node_redis.git package that I wanted to install with NPM.

c:\node>node ./npm install -g git://github.com/mranney/node_redis.git

...which produces the simple one-line output as a result of a "successful" package install:

redis@0.7.1 c:\node\node_modules\redis

Ah, that is better!

Now, on to what this same issue presents like under Linux...

Node Package Manager Linux Behavior and NPM Error Messages due to using wrong .git file URL

I have been using Node (Nodejs) under OpenSuse 12.1 x64 KDE.  When I first encountered this issue under Windows, I quickly jumped over to my Linux VMware virtual-machine to see if for some reason it was a Windows-implementation-only issue (since, Node and NPM have been more mainstream on Linux, and the NPM under Windows is considered "experimental" yet).

This quick test under Linux helped me see the error in my ways quickly, since I encountered a similar mess of misleading error-messages spewing forth from the node-package-manager when I copied and pasted my npm install command to Linux and executed it:

~/node> node ./npm install -g https://github.com/mranney/node_redis.git

Yes, it failed similarly to the Windows NPM version, but with an error message (pasted here) that sure made it appear like the the "tar" (unpack) command failed for some inability to work with the temp .tgz file created as part of the install process...

npm ERR! Failed unpacking /tmp/npm-1322603639405/1322603639405-0.840085425414145/tmp.tgz
npm ERR! couldn't unpack /tmp/npm-1322603639405/1322603639405-0.840085425414145/tmp.tgz to /tmp/npm-1322603639405/1322603639405-0.840085425414145/contents
npm ERR! Error: `tar "-zmvxpf" "/tmp/npm-1322603639405/1322603639405-0.840085425414145/tmp.tgz" "-o"`
npm ERR! failed with 2
npm ERR!     at ChildProcess.<anonymous> (/home/mike/node/npm/lib/utils/tar.js:217:20)
npm ERR!     at ChildProcess.emit (events.js:70:17)
npm ERR!     at maybeExit (child_process.js:359:16)
npm ERR!     at Process.onexit (child_process.js:395:5)
npm ERR! Report this *entire* log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <npm-@googlegroups.com>
npm ERR! 
npm ERR! System Linux 3.1.0-1.2-desktop
npm ERR! command "node" "/home/mike/node/npm" "install" "-g" "https://github.com/mranney/node_redis.git"
npm ERR! cwd /home/mike/node
npm ERR! node -v v0.6.2
npm ERR! npm -v 1.0.106
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/mike/node/npm-debug.log
npm not ok

Having performed successful node module installs with NPM under my Linux host already, I knew that I must be missing something obvious.  And, without any help at all from those meaningless error messages, I finally saw that I pasted the "https://" address of the github node module address instead of the git:// version.  Fixing this mistake, the module installed just fine under both Linux and Windows.

This revelation (about my mistaken "https" vs. "git" prefix), coupled with my frustration at misleading NPM error messages, led to the writing of this blog in case anyone else runs into this issue and gets misled by utterly meaningless error messages.

I hope someone working on the NPM (package manager for node) will eventually make time to implement proper condition-testing (and meaningful error-reporting) to test the format of command-line parameters as important as the actual package/module URL if the format makes such a tremendous difference in results.  Had this software simply tested the URL for a valid format, and told me "sorry, you must use git:// prefix" or suggested another equally effective alternative, that would be ideal and would save myself, and surely others, from wasting time with misleading error messages.

Redis (Redis.io) Database via Node

As an aside, I can report that I am able to access my Redis (NOSQL) DB from both Linux and Windows versions of Node via the node_redis module's functionality from within Javascript.  I hope to add more blog material about my experiences with installing and using Redis with Node at a later date.

Saving Money in Information Technology and the Data-Center

Green Technology : Energy-Savings Techniques

Saving Money by Reducing Energy Consumption

I was just reading the Computerworld top 12 green-IT organizations list for 2011, and after many pages of reading, it was clear that certain energy-savings strategies were quite common across the range of companies featured in the article. I am all for saving energy, as this not only reduces the carbon-footprint of an organization, but it also saves money!

Some of the techniques noted should spur ideas about how you and your technology organization can save money by focusing on being "green".  Here is a summary of how the top-12 companies made substantial strides in reducing their energy consumption through energy-savings initiatives that led to real and measurable financial cost savings too:

• Virtualization: this is a significant savings opportunity for many of the companies featured in the roundup of green IT organizations.  Virtualization is moving beyond server and network virtualization and into the virtualization of desktops also, for some of these companies.  The ultimate objective of this push into virtualization is to consolidate (computing) workloads onto as few machines as possible and achieve very high physical-machine utilization rates.
  
  Allstate Insurance was featured for its ability to reduce its total computing server and device count by 3,000 units while realizing a cumulative energy-reduction of nearly 40% in the past several years. NBC Universal virtualized 60% of their physical servers and shut down 2,000 physical machines. Northrop Grumman posted substantial energy savings through widespread virtualization also, and they are considering thin-client desktops for further savings. Citigroup has gone as far as to require all new servers be virtual, unless physical servers are justified; this has reduced their power and cooling requirements by 73%.


• Cooling system efficiency: modern data centers pack incredible amounts of computing power into confined areas, and as such they require substantial cooling systems (i.e., air-condition).  Kaiser Permanente was a focus company in this area, as they achieved energy-consumption savings across their three data centers that are nothing short of incredible, "cutting an eye-popping 7.2 million kilowatt-hours of power from overall data center operations [this year] -- and over $770,000 from power budgets."
  
  How did they do it? They focused on everything from sealing up air leaks (which provided the biggest win) and a sophisticated real-time monitoring system for measuring cooling system efficiency.  They were able to reduce cold-spots in the data center (which are signs of inefficiency) and avoid overprovisioning power distribution and cooling infrastructure in general.  NBC Universal was also mentioned for having implemented similar smart power distribution units and rack-level environmental and power metering sensors, allowing them to increase rack densities by as much as 200%.


• Raising ambient temperature (in the data center): the IT group at KPMG was featured for its efforts, which included "raising the ambient temperature in the data center to improve efficiency by more than 5%, raising the temperature of the water in the cooling tower to improve efficiency by 5%".  Raytheon was also featured for how IT cut energy use in each telecommunications closet by 30% when it raised the temperature by 10 degrees Fahrenheit to 75 degrees
  
  One related news bit I did not see mentioned in this Computerworld article was how Dell has recently announced that some of their servers (currently the R610 1U and R710 2U varieties) are capable of running at what is essentially typical outdoor-ambient-air-temperatures (in nearly all of the USA, and nearly all year round).
  
  These run-hot-capable Dell products (and Dell, the company and the stock: NASDAQ:DELL) are worth watching, as the energy-savings implications are huge for these high-operating-temperature Dell servers. There are also Dell storage arrays, switches, and power-distribution units that are certified to run hot.  I expect more computers (especially servers) will end up being certified to run hot like this, which could vastly reduce cooling costs for IT facilities as ambient air temperatures could approach outdoor levels (as such, lower cost air-movement by way of fans vs. chillers could perhaps become the norm for much of any year). 


• Blade-Server technology: KPMG was again featured for its migration to blade server technology, with "the average blade server consuming about 50% less power than a comparably configured rack-mounted server".  Other firms were mentioned for their push to increase rack computing densities through such means.


• Data-Compression and storage de-duplication to reduce physical hardware required to house company data.  The finance firm State Street was noted for its efforts in this area where they reduced storage use by between 40% and 50%.  This should lead to about a similar percentage savings on both power and equipment costs related to storage.


• Private Cloud Computing and High-Performance-Computing (HPC) Clusters were mentioned by some of the top-12 energy-saving firms. "Baker Hughes created a high-performance computing (HPC) cluster that incorporates wake-on-LAN technology. In this environment, machines are turned on -- or "woken up" -- via a network message, and the company can wake up machines for use in the HPC cluster as needed. This setup uses 40% less energy than a dedicated HPC pool."


• Solar Panels have been installed by some of these firms (KPMG), and surely other type of alternative-energy will become more common in data-center planning in the future.  I didn't see any mention of things like backup power based on modern fuel-cell technology or micro-turbines and such, but I would expect that some of this is in play for various firms.


• Telepresence to reduce travel. Various firms, including KPMG and Allstate, were mentioned for taking steps to reduce employee travel, not just locally, but nationwide inter-office travel as well.  Technology for streaming video and web-conferencing are employed to make this possible.
  
  To me, it seems incredibly obvious that one of the largest "green" moves any firm can make is to enable workers to work from home whenever possible.  Though, simply put, I do believe that many workers are just utterly incapable of managing their time and remaining focused while working at home as they let all the distractions of home interfere; this explains why, with all this modern technology that would make a vast number of jobs possible from home, there still is not widespread work-from-home opportunity.  It will probably require $10/gallon gasoline to make a substantial change in this area.

SVG onload event not firing : Firefox bug / feature with Shortcuts

FireFox not firing SVG onload event

(Windows) Shortcut handling to blame...

I do a fair amount of work with SVG (Scalable Vector Graphics) images / files that contain embedded JavaScript for various event-driven interactive-SVG components. The onload() event, within SVG files, is something I regularly use too. Today I ran into a strange "feature" or "bug" that shows up in Firefox but not in the Google Chrome / Chromium browser — related to this onload event in an SVG document.

I use Chrome as my default browser, especially because I like the included Developer tools a lot, so most initial testing of my web-page HTML, SVG, and JScript code takes place in Chrome before I move onto testing in other browsers (like FireFox). I have been working on my custom SVG RAD Components (tis' what I currently call them), and because I use one particular .SVG file for the main "test rig", I kept a Windows Shortcut on my Windows 7 desktop for a quick reference to that .SVG file.  I just click the shortcut to launch my SVG "application" (in Chrome) or drag the shortcut into a Chrome tab, and that works just fine.  Ah, but not in FF!

FireFox apparently does not resolve shortcut properly if dragged into browser

Being a creature of habit, when I was ready to test my latest SVG file and Javascript code within Firefox (using version 7 currently), I dragged my Windows shortcut (to my SVG file) onto the FF browser and poof... it seemed to load the SVG file, but my onload() event code simply failed to run.  I would have sworn I did this exact same drag-to-load (my SVG) with prior versions of Firefox successfully, but either way, it is not working now.

So, I loaded the page again via drag-and-drop of my Windows-shortcut to my SVG file, but this time with Firebug running (debugger / developer tool).  I quickly see that an error is being generated whereby the event-code referenced in the onload() event was shown as "myOnloadFx is not defined" within the onSVGLoad() code in FireFox (evt=SVGLoad). Clearly something strange was going on here, as this code "works" and has worked in Firefox before.

I played around with the code inside the SVG file a bit, and moved the onload() code from the opening SVG tag's onload="myOnloadFx()" to an inline-script (using <script> tags) just before the SVG's closing tag... and, the problem persisted.  So, what the heck?  After wasting more time on this than I ever should have, I then decided to go to the directory in which the SVG file really existed (vs. using the Windows shortcut to open it), and I dragged the .SVG file onto the Firefox window where it opened fine and ran the onload() event code just as Chrome did.  So, the shortcut-dereferencing/resolution is apparently to blame.

FireFox : want your SVG Javascript onload to fire? Do not open the SVG by dragging a shortcut onto FF

Now I know.  Note: the code executed in my onload() event was in an external Javascript file that is "included" in the SVG by way of code like this: <script type="text/javascript" xlink:href="myExternalSVGcode.js"/>  

Firefox is not properly converting, storing, and subsequently referencing the proper file-locations for included-code like this, but is instead looking in the directory where the shortcut appears (in my case, the desktop).  I confirmed this to be the problem simply by moving a copy of the Javascript (referenced) file onto the desktop along with the Shortcut (.lnk) and voila!  It "fixed" the issue.  UNREAL. 

I have not tested other types of scenarios where this could be a problem, and it is unlikely most people will ever encounter this unless they do software development (and perhaps even just SVG/Javascript with included external javascript files).  But, just in case, I figured I would post my notes here for anyone else that may encounter this weird onload behavior in Firefox.

Embarcadero Delphi XE2 / FireMonkey Review Findings : Not Ready, Incomplete

Delphi XE2 : Reviewers / Users Not Pleased

UPDATE: Delphi XE3 release date is here, and I have posted a blog entry about Delphi XE3 New Features; some of the concerns with XE2 / FireMonkey may be resolved with FireMonkey2 / XE3 improvements. 

---

I am a long-term user of the Embarcadero Delphi (formerly Borland, Inprise, and Codegear branded Delphi) RAD (Rapid Application Development) IDE, programming language (which is object-pascal) and VCL (Visual Component Library).  Recently I wrote a blog about the Exciting New Features in Embarcadero Delphi XE2 and how much I was looking forward to putting the new FireMonkey components to use in particular.  Also, while reading this, keep in mind the fact that (for me) FireMonkey was the one thing that was going to keep me interested in this product for my Windows development platform of choice.

Well, here is the simple fact of the matter: Delphi XE2 / FireMonkey is not ready for prime-time.  I have discovered that I am not alone in this opinion, and it appears that Embarcadero has done itself no favor by releasing what many consider an unfinished product.  Perhaps I am premature in predicting the final demise of this platform and product, but that is essentially what I am doing: it is over... at least for me.  I have used Delphi since version 2.0, going all the way back to 1995, and have been a loyal and avid fan of the development language, IDE, and components.  But, I am 99% sure that I am finally calling it quits. Here's why...

Poor and Missing Documentation

I have totally had it with the total CRAP documentation that has come with Delphi since ever since Delphi 7.  I have heard promises of improvements for years, and with every small improvement comes a host of missing documentation, bad or missing hyperlinks within the documentation, a poor (and god-awful slow and inefficient) documentation-browsing interface, etc.  I long for the old days where I could quickly find all the help I needed and quickly navigate links to related information; those days are long gone.

I just can not take it anymore.  It is absolutely inexcusable how pathetic the documentation efforts have been, regardless of which company has owned the rights to Delphi (Borland, Codegear, Embarcadero).  Clearly documentation is barely as step above an afterthought for these companies lately as they probably consider it nothing but an "expense"; never mind that this expense is what leads to a product I actually may want to purchase!  With all sorts of new features simply nowhere to be found in the documentation, my level of frustration is epic.  I have seen as good, or perhaps better, documentation efforts for open-source projects (you know, all those projects that are notoriously lacking documentation). 

FireMonkey Bugs, 64-bit bugs, Missing Features

I am not going to re-hash everything that people are experiencing with FireMonkey, as it just furthers my frustrations with this incomplete and buggy product.  But, here are some of the things that are worth noting, as pointed out on blogs like Delphi Haven and AnalogMachine:

• the omission of Actions and action lists
• aside from the very basics, Keyboard handling is crippled
• Property/method reference documentation (as I more than hinted at earlier)
• TMemo has serious bugs.
• Substantial differences (and missing capabilities) in FM form-designer behavior vs. VCL-forms designer behavior.
• the 64-bit Compiler has some serious bugs.
• And many more...

Delphi is now looking more and more like a total dead-end for me and certainly will not help my career in the least bit; I am no longer willing to wait for bug-fixes to back their technological improvements that ship without proper testing.  And, I do not want to continually pay for the right to deal with such poor quality.

Delphi XE2 Update-1 and Update-2

Embarcadero has released the first couple "updates" (Update 1) for Delphi XE2 and (Update 2) for Delphi XE2 that will supposedly begin the process of correcting some of the deficiencies in this product. But, I am at the point I do not care. Why? Because we all know (based on past behavior) how this will go... they will issue approximately 4 "updates" (i.e., BUG FIXES) to the product during its life-cycle and before Delphi XE3 is released, at which point you then will be forced to PAY AGAIN for what should be free and continued bug-fixes!

And, like darn near every single "release notes" published in the past 5 years by these folks, this XE2 Update 2 release-notes document also has screwed up hyperlinks to the supposed "list of fixes".  Currently, in the "General" section of this document, see where it says (I cut/pasted this from their site):

"For a complete list of the specific problems fixed in this update, see the "List of Bug Fixes in Delphi and C++Builder XE2 Update 2" at: <a href="http://edn.embarcadero.com/de/article/40984/">http://edn.embarcadero.com/de/article/40984/</a>"

Well, guess what folks... the above quoted HTML really links to some OLD Delphi XE Update 1 list of fixes and even points to the German location (/de).  THIS IS SO TYPICAL!  AGAIN, WHERE IS THE QUALITY-CONTROL!?  I really would have liked to see what is truly "fixed" in this XE2 update.  Ughghgh!  Do we now have to post a QC (Quality Central ... heh, "quality", yeah... ok) entry to report this messed up URL?  Again, this is TYPICAL and has been the case for years.  I think it is done ON PURPOSE since they really have no list of updates prepared!  But, who is dumb enough to keep doing this and keep peeving their customer base?!

This practice of shipping bug-ridden, unfinished software products is common among MANY software firms these days... they put out an unfinished/buggy product, patch it a few times for "free", then force users to purchase yet another "upgrade" to get any future fixes/improvements regardless of how strong the case is that there are still substantial bugs in the product you PAID for!  It gets old, and it gets very darn expensive! And, I have seen long-standing substantial bugs go unfixed through multiple major-product-versions/iterations (Delphi and their help-system clearly have had long-standing substantial issues).

Delphi XE2 : the end of the road for my Delphi use

For the reasons laid out already, and for the added fact that there are VERY FEW Delphi Jobs / Contracts available anywhere these days, I am leaning toward totally ditching closed-source development tools like Delphi.  There are some great free and open-source products and technologies out that I am going to be evaluating in depth and preparing to move to.

I am likely go to just HTML5 / SVG / JavaScript wherever possible, and I may totally ditch "native Windows applications" except for those situations where I absolutely require the performance and sophistication I can only achieve with compiled code.  I have prior versions of Delphi that I will continue to use to support some of the systems I have written for my own use of course, but I also plan to begin the inevitable migration to other technologies.  I am not looking forward to migrating my systems include lots of SQL-Server interaction (using VCL dbExpress or ADO components), but such is.

Although I believe Delphi has provided a super-productive development environment and language for developing Windows applications over the years, I also see the writing on the wall: Delphi is a niche tool whose niche continues to shrink as people move to web-based everything.  And, Embarcadero is not helping Delphi's cause when they release buggy unfinished software with equally pathetic documentation.  Perhaps I will again assess Delphi when the inevitable Delphi XE3 appears on the scene, but for now, I am avoiding any further investment of my time and money into this product.

UPDATE (January 2012) for anyone still interested in Delphi XE2: Embarcadero has released Delphi XE2 Update 3 and has put a new Delphi XE2 ISO Image with Update 3 online.  I have not taken time to download and install it, though I did read through the release notes of fixes in Delphi XE2 Update 3, and it appears they are slowly working through the massive pile of major errors and issues.  Surely it must be "better" than the initial premature release of XE2. But, I am still a major skeptic and will only consider a hands-on Delphi re-evaluation when Delphi XE3 is released -- and, Embarcadero had better take time to ensure a much higher quality product if they are to convince me to ever use their development tools again.

Embarcadero Delphi XE2 New Features of Interest

Embarcadero Delphi XE2 Review of New Features — VERY Interesting Features!

I have been anxiously awaiting the official Embarcadero Delphi XE2 release now that this RAD (Rapid Application Development) IDE and Component-Set are poised to bring about some of the most exciting changes in Windows (and cross-platform!) application development seen in a long time. Yes, Embarcadero (formerly Borland, Inprise, and Codegear branded) Delphi is finally stepping up the application-development game and introducing some exciting technology to address shortcomings in modern MS Windows business-software-applications development.

Although many will cite the most interesting new features as those being support for cross-platform development, Windows 64-bit, Amazon Cloud API, Native iOS support, for me the obvious "killer feature" is the new vector-based UI-development component suite and technology called "FireMonkey". The reason I choose this as the "killer feature" is because 1) I have felt that Windows-UI development has been rather "stale" for years when building *native* (compiled executable applications), and 2) this technology is what certainly makes large portions of other features (like cross-platform development) even possible, and finally 3) I consider this technology capable of building real mission-critical business applications

Delphi XE2 FireMonkey : Vector-based User-Interface Development

Delphi XE2 FireMonkey: Could it be a Disruptive Technology?

FireMonkey is the moniker Embarcadero has applied to their new scalable vector graphics based Graphical User Interface (GUI) framework that is leveraging the capabilities of modern GPUs (Graphics Processing Units) for hardware accelerated cross platform GUI’s. If that sentence did not make it obvious: this is BIG, people!

FireMonkey really could be the disruptive technology we have been waiting for with regards to developing compelling UI's for business applications. It is about time scalable vector-graphics melded with mainstream business applications (and yes, I am aware that Flash and Silverlight are vector-based technologies; I just still do not consider them to be something I want to build any large-scale enterprise applications with).

If Embarcadero is successful at marketing this vision of how modern UI's should be developed, we could be on the cusp of a huge shift in *native* application UI technology (note: I consider HTML5/JS advancements also disruptive, but in a different way and for a somewhat different target-market).

So, vector-based User Interfaces (UIs) are soon to be a reality for Delphi developers and the software applications they create (and therefore Microsoft Windows environments), but FireMonkey holds even more promise than simply modernizing our UIs — this technology is what will allow resulting UIs to now be cross-platform capable. FireMonkey provides UI elements that will ultimately look the same across the various deployment targets: 32-bit Windows applications for Windows 7, Windows Vista and XP and Server OS's... 64-bit Windows applications for Windows 7, Windows Vista and XP; Server 2003 and 2008... and even Apple OS X 10.6 and 10.7 applications and iOS 4.2+ applications. What? Apple? Since that cross-platform development tool news is going to be of substantial interest to many people, I will discuss that later in this blog;likewise you have taken note of the 64-bit reference which will also be discussed in more detail.

Delphi XE2 FireMonkey: Where did it Come From?

FireMonkey is based on VGScene/DXScene, which was created by KSDev (Eugene A. Kryukov), and then purchased by Embarcadero quite recently (late 2010 - early 2011). KSDev sold VG/DXScene as a VCL component package prior to the acquisition, and KSDev's final release on 1/13/2011was considered "feature complete".

KSDev marketed their components as: "a Delphi VCL library for WPF-like framework with advanced controls, styles, graphics and effects", with the core functionality being built around a powerful vector-engine (similar in concept to how Adobe Flash works) with modern features like real-time anti-aliased vector graphics, resolution independence, alpha blending, gradients and special visual filling,etc.

I actually looked into using DXScene / VGScene back over a year ago when I found myself, once again, thinking how outdated my Delphi GUI applications looked and how utterly annoying I found it that native executable Windows applications (in general), and the UI elements that made up these GUIs, did not SCALE easily when switching between various screen-sizes and pixel-densities, and that there was no easy way to give my applications the refinements to look more "modern" without purchasing a bunch of third-party controls. And, purchasing third-party controls to address the issue of modern "look" still did not resolve the issues with easy scaling of the UI-elements.

After looking at the KSDev stuff, I actually opted not to embrace their components for a few reasons. First and foremost (at the time) I considered them to be a high-risk "niche" component-set that I was unwilling to risk building mainstream applications for my customers with. I have seen all too many Delphi VCL component sets wither (think Rave Reports) and/or completely die-off over the years, and even if source-code is available, many component sets are just so specialized that it would take far too great of an investment to continue to use them in the event the developer "gave up" on them or failed to produce necessary bug-fixes and so on. KSDev had a neat thing going with their components, and thankfully Embarcadero has picked up that work and provided the credibility and reassurance I need to actually implement business-applications using that technology now in Delphi XE2!

Delphi XE2 FireMonkey: Is it Like X, Y, or Z?

This cross-platform application framework uses GPU-accelerated vector graphics to render UI elements, D2D/D3D (Direct-2D / Direct-3D) on Windows, and OpenGL on OSX. You can think of it as similar to Silverlight OOB or Jupiter (the new “application model” for Windows 8), or even Adobe Flash. When I consider the goals of Microsoft's "Jupiter", I have to wonder if perhaps FireMonkey is essentially the same thing... here is what a ZDNet article from early 2011 described Jupiter as:

Jupiter is going to be a new user interface (UI) library for Windows, built alongside Windows 8. It will be a thin XAML/UI layer on top of Windows application programming interfaces and frameworks for subsystems like graphics, text and input. The idea is Jupiter will bring support for smoother and more fluid animation, rich typography, and new media capabilities to Windows 8 devices.

Hmmmmm... sure sounds quite similar with regards to the end-result (the UI people see), though thankfully the FireMonkey implementation is not a pile of XAML and over-complexity that Microsoft always seems to come up with.

Instead, FireMonkey uses the familiar Delphi (object pascal) language and VCL (Visual Component Library) paradigm for its implementation, and compiles to native code. To me, being able to work with FireMonkey is just like working with any other VCL components, and honestly anything that keeps my from having to learn yet another Microsoft UI-technology-of-the-day is a plus (I just can not deal with XAML).

FireMonkey: Will Microsoft FUD Bury it Before it Takes Hold?

At least part of me is concerned that Microsoft will somehow work its usual FUD (Fear, Uncertainty, and Doubt) campaign against Embarcadero (with regards to FireMonkey) as they work feverishly to bring their own "Jupiter" vision and Windows-8 to market.

For all you LONG-TIME Delphi developers, do you remember how the Visual Basic vs. Delphi thing played out over a decade or more? Clearly Borland was (what should have been) light-years ahead in the RAD IDE and component-based Windows development tools/language space (especially in OOP that people could understand; unlike C++ which dominated mainstream "real" Windows apps prior to Delphi), but Microsoft worked very hard to convince developers (and corporate management) that investing in Delphi was a bad move... that Delphi was too risky,.. all the while working to "catch up" with Visual Basic and push that as the "solution" to corporate UI-development needs. I have used Delphi since version 2, and the fact is, Microsoft was not even remotely close to having anything as capable until perhaps the days of Delphi 2006.

As we all know in retrospect, Microsoft's strategy worked in a BIG way and only after burying Delphi and relegating it to the niche-market they fabricated through FUD did MS create a semi-decent (though wildly bloated) component set of DotNet and the reasonably nice C# language (which is clearly based on Delphi to some extent). I am concerned that somehow Microsoft will wage such a war again if they decide Embarcadero is a "threat"; or, do they even need to?

The fact is, Microsoft's decade+ campaign of marginalizing otherwise promising, and even superior, development languages and technologies has been so successful that Embarcadero has a monumental task ahead of it: convincing mainstream corporate developers to actually embrace this technology. Good luck with that!

There are so many "competing" priorities pulling at corporate IT-folks and budgets that I see this as a battle that is going the be VERY difficult to win without some serious willingness to put some flesh on the line and suck up some losses while doing "a Microsoft" and dumping the product out there en masse, and even at a potential loss, to foster widespread adoption so as to gain the all-important "critical mass" necessary to propel the product forward and create a self-sustaining win vs. a self-fulfilling-prophetic-loss situation (for lack of developer density, etc).

FireMonkey is also up against HTML5/JS hype, up against Silverlight/Flash and the forthcoming "Jupiter", and so many other competing technologies... how is it going to gain traction? When we (developers) search sites like Dice.com and see essentially ZERO postings for Delphi developer jobs (compared to oodles of C# or Silverlight or HTML5/JS jobs), what are we to do? 

Embarcadero best be thinking long-term and be willing to take a bit of "a hit" (financially) to gain a foothold, or the simple fact is: the niftiest technology in years for UI development may make little difference to market penetration and adoption. Get your marketing/sales team (especially the latter; since "marketing" is sales without responsibility for producing revenue) ramped up NOW Embarcadero, and have them start working some serious deals with software developers to get them to use this product! OK, enough said... on to more about this tech...

FireMonkey: Embrace it, and Embrace Changes to Your Existing Applications

FireMonkey is an entirely new framework for UI-development, and as such, it is incompatible with your current/traditional VCL-based UIs and you are not going to be having co-existence in the same application (i.e., if you want to port an existing application UI to the new FireMonkey technology, you will have to rewrite your GUI code).

This sure sounds a bit overwhelming, but I really think this gutsy move by Embarcadero is what will actually give FireMonkey a fighting chance — the technology is not encumbered by the burden of legacy support! This makes the implementation MUCH cleaner — we all can attest to how much we welcome the opportunity to write an application or component "from scratch" as compared to modifying a many-revision-old, widely used (and thus many possibilities to "break" something), piece of code. I expect this new code to be architecturally solid and much more ideal thanks to separating it from the older UI-VCL components.

FireMonkey Components are Containers

You will also have to get used to a bit of a paradigm shift with regard to how components are assembled, and I think it is another shift that is for the better and about time: FireMonkey components are all containers, meaning you can embed any component inside any other component. When you think about it, this makes total sense.

Something as simple as a button-component is composed by assembling 9 components that, when put together, produce what looks and behaves as a Button should. A FireMonkey Button consists of: a TLayout component to organize all control layout, (3) TRectangles for border, background and foreground color, a Label represents for the Button text, and then a group of four additional components (two each, for animation and effects).

The animations are going to give us the visual mouse over/out on the button (like we are used to seeing on websites for years), and the effects can occur on events like button-press, onfocus, etc and make even niftier things like "glow" effects happen and so on. This type of animation/effects ability is present throughout all of FireMonkey components thanks to the way these containers and component-buildups can be implemented.  I look forward to using this to "modernize" the look and feel of my applications, though we all need to keep in mind that this could be over-used quite easily.

You may also want to think about how to standardize the look/feel of your application elements, and thankfully FireMonkey implements what is a parallel to CSS Styles through their own FireMonkey "Styles". I am not yet sure how far these Styles can be pushed, but I am hopeful this first version is good enough for most things. I think about how CSS has gone through a lot of change as we push into CSS3 now, and I wonder if future iterations of Delphi XE3, XE4, etc will be extending the power of their own Styles just like how CSS keeps growing its abilities.

In some regards, these Delphi FireMonkey styles are quite a bit more advanced: you can implement things like blurs, animations, and so forth, via styles. Again I have some concern about pushing UI-glitz TOO far, but, no matter what, Styles should make standardizing and quickly updating the look-and-feel of applications a LOT easier!

Perhaps FireMonkey applications will be the advertising-force Embarcadero needs to gain further recognition: when users and developers start seeing native applications that are simply stunning, they may start to ask "what is that written in?" This could be a positive thing, but I also can imagine some applications getting so ridiculous with animating every last aspect of the UI that, when that previous question is asked, it will be with a bit of disdain or ridicule.

Hopefully we all use this power wisely :)

What about Non-Visual Components?

You may already be thinking: what about all my VCL components I use like TList, TStringList, etc.

Have no fear: these non-visual components will remain the same as what you are used to and will also be usable from your FireMonkey-based-UI applications. The fact is, if you have already done a decent job of separating your UI-implementations from the underlying event-code, database-interaction, and such, you may not have TOO difficult of a time updating your applications.

You are not going to have "data-aware" components like TDBMemo anymore under FireMonkey, but of course there is an alternative way of going about this. The new "LiveBindings" within the FireMonkey framework allow you to connect any type of data to any UI or graphical element in VCL / FireMonkey; consider this a mechanism for creating "live" relationships between objects and also between individual properties of objects. It has some serious potential!

I am excited by this feature, and look forward to seeing how far I can push these live-interrelationships. LiveBindings are going to allow you to do thing you can not do with existing data-aware controls too. And, LiveBindings are *not* just limited to FireMonkey controls (i.e., there is support for this technology in the "old" style VCL too as part of Delphi XE2 updates). You will be able to do things like bind the "Caption" property of a TLabel to the Field-values in a dataset (or the column-name, etc), and much more.

Since the "bindings" are accomplished using an expression-engine (vs. just simple hard-coded bindings) you can bind on evaluated-values. E.g., bind your label control's caption to an expression like TDBColumn.DisplayName + " column value is:" + dataset.field.valueAsString (pseudo-code used for example). You get the idea. It really is powerful.

But, that is not all... If you choose, you can implement bi-directional property-to-property bindings (which, sure sound like data-aware-like functionality). This bi-directionality implies something somewhat profound: it should be possible to consolidate UI-element-frameworks to no longer require those TDB...versions of each control (i.e., remove the need for "data-aware" versions of each control), since something like a Label can be bi-directionally "bound" and suddenly be that data-aware-control.

This is going to take some hands-on experience to get used to, but it is a significant step forward (and, should bring writing "data-aware" custom controls into the realm and reach of many more developers; I say this because I have always found writing TDBxyz data-aware custom components WAY too difficult!).

Note: I have read that the expression engine used by LiveBindings is available to us in our programs to evaluate any ObjectPascal expression dynamically at runtime; this should make for some interesting neat applications too!

Cross-Platform Native Applications using Delphi

OK, this topic certainly deserves some attention, especially from all of us that can still remember the days of Kylix, which was a nifty idea but one that failed miserably for all sorts of reasons (one being the simple fact it was not maintained at all after early releases). Well, with that memory pushed aside, let's think about the prospects of true cross-platform NATIVE-code deployment again.

As mentioned earlier, FireMonkey provides UI elements that will ultimately look the same across the various deployment targets: 32-bit Windows applications for Windows 7, Windows Vista and XP and Server OS's... 64-bit Windows applications for Windows 7, Windows Vista and XP; Server 2003 and 2008... and even Apple OS X 10.6 and 10.7 applications and iOS 4.2+ applications. In addition, there is some speculation that Android support and Linux will be forthcoming soon after the release of Delphi XE2 (hopefully as a free update!!)

The IDE Runs Only On Windows : but, you can deploy to other targets

It is not surprising that the Delphi XE2 RAD IDE runs only on Windows, though part of me wonders if Embarcadero will get around to converting the IDE to be FireMonkey-based (if even remotely possible?) and make the IDE run on any target-platform. Regardless, for now it is Windows only (as it always has been; aside from Kylix), and we developers will have to go through a few extra steps to compile and deploy applications to the Apple targets.

Delphi for Apple OSX/iOS

From what I have gathered via online discussions (I have not tested the Apple deployment stuff at all, nor do I have much initial concern for it even though long-term I expect to support Apple targets), Embarcadero / Delphi is apparently relying on the FreePascal Compiler (FPC) to compile code for deployment to other (non-Windows) target operating-systems. 

The FPC compiler will use the same source-code you have written for your Windows-based applications (that Delphi's compiler used to generate Windows binaries) and the FPC will generate binaries (i.e., native apps) that can be run an Apple / Mac computer and/or iOS device (i.e., single collection of source code yields multiple-platform-specific binaries, thanks to some FPC help).

There are also significant limitations with what all can be simply recompiled and deployed to the Mac. I am under the impression that outside of FireMonkey, substantial portions of the VCL will not be available on the Mac yet (I may be wrong). And, I really can not imagine some things EVER being supported on the iOS/OSX platform (especially some of the "native" database-access stuff).

You will certainly need to use FireMonkey for any UI you plan to have run on the Apple side of things, but in addition, I suspect there will be all sorts of other caveats regarding what will and will not "port" directly simply via a recompilation. Again, I see the Apple thing as a longer-term possibility for me. I'd be more intrigued with Linux deployment immediately (since I have Linux running in a Virtual Machine or two). Time will tell. I look forward to seeing what people are able to achieve on the Apple platform with Delphi XE2.

Delphi Applications for Cloud / Hosted Scenarios

I came across a sentence online somewhere stating that "Delphi and C++ applications can be deployed to Amazon EC2 and Windows Azure, with support for Amazon Simple Storage Service API, Queue Service, and SimpleDB." This is interesting, but I really do not know exactly what was needed to support this, and so far, I have not had the need for this.

Delphi XE2 64-Bit Support

Native 64-bit Windows applications are something that quite a few Delphi developers have clamored for over the past couple years, and apparently they are getting their wishes fulfilled. Delphi XE2 is to include support for Windows 64-bit machines, including a debugger and deployment manager. And, it looks rather easy to deploy an application as a 64-bit applicaition.

In the Delphi XE2 Project Explorer you will see a new node under each project where you can choose your "Target Platforms". By default, your existing projects are going to have a "target platform" entry that is ideal for deploying to 32-bit Windows. Next, you can add your new 64-bit Windows platform target node to the tree, select it, recompile, and voila!, you have a 64-bit executable.

I do not know how many developers really "need" 64-bit capabilities (like is necessary for addressing very large blocks of memory or working with 64-bit integers, etc), but now the capability is there and Delphi need not be considered lacking in this regard. You will have to do some (most likely) minor "code review" to make sure you to not have any code that is for some reason only 32-bit-safe; e.g., you are doing some bit-level manipulation shifting bits around in INTs, doing crazy things with pointers, etc. I do not expect most people will have significant work to do in this regard prior to 64-bit compiler and deployment use.

Delphi Reporting Components Update : Finally!
Goodbye Rave Reports (Junk!)

OK, I could not obtain 100% confirmation of this quite yet, but rumor has it that Delphi XE2 will include FastReport VCL 4 RAD Edition reporting tool — whether true or not, the fact is I refuse to invest ANY more time using Rave Reports (what a buggy pile of @#!@ that is an embarrassment that needed addressed; Nevrona's pathetic "support" and glacial pace of resolving any issues and bugs caused me and many others to become utterly fed up with the product and move elsewhere).

FastReports surely has to be a better option by a long-shot, as it is actively maintained and developed. Compare that to how Nevrona can not even update their *website* for years on end. I can not believe how long it took Embarcadero to move past Rave Reports... perhaps they made the stupid move (or Borland did) of signing some longer-term contract with Nevrona without any sort of "out" for them not meeting certain quality criteria, support criteria, etc. Who knows. But, I am excited by the prospect of having a good reporting tool (by default) included with Delphi!

Other New Features in Delphi XE2 Worth Noting

More details will emerge quite soon. In fact, I am supposed to listen to a Webinar about the product-launch tomorrow hopefully a near-term Delphi XE2 release-date is to be announced. And, I also hope the RTM (i.e., final, release-ready) version of Delphi XE2 is truly "ready" and not full of a bunch of annoying bugs.

My guess is that like most recent releases, there will be an update-pack available for it nearly as soon as it is officially "released"; hopefully it is solid enough to be truly prime-time ready. I have quite a few Delphi applications I want to update to take advantage of these new features ASAP.

I am not a big "DataSnap" user, but this release is supposed to have a fair amount of updates to that functionality. There are components and functionality related to that new "Cloud" stuff like TAzureQueueManagement, Amazon Simple Storage Service API, Amazon Queue Service API, Amazon SimpleDB API, and so on. I think the Documentation Insight is new (Delphi XML documentation tool) too.

Either way, there is a LOT of new stuff packed into this XE2 release, as already discussed. To me, the FireMonkey stuff alone is a HUGE chunk of functionality and makes me quite eager to start building some fantastic XE2-based applications leveraging these features.

Delphi XE2 — CONCLUSION: Enterprise Applications are Poised for a Major Update

As reported in this brief SD Times article and interview, Michael Swindell, senior vice president of product management for Embarcadero, seems to be clearly positioning Delphi XE2 and FireMonkey where I see it making the most sense: business applications:

“We know where we should be going with the experience of non-entertainment applications,” he said. [in reference to the fact that FireMonkey ships with about 200 user-interface controls that include GPU-powered scalable vector and 3D effects] 

[...] 

Swindell emphasized that FireMonkey is focused on heavy-duty business applications—not entertainment or advertising sectors, where rich Internet applications already are strong. To that end, FireMonkey introduces a feature called Live Binding, which lets developers bind any UI control or graphical element to any data source, he said. Native CPU application execution and data access allow FireMonkey applications to perform at a very high level, he added. 

[...] 

“We saw this as a gap and as where applications need to go,” Swindell said. “Companies continue coming out with 1990s-style Windows Forms applications and rolling their own frameworks. There hadn’t been anything out of the box to get [developers] there quickly and with a lot of power.”

I could not agree more with that concluding quite about the "gap" that existed. Being a business software developer, I am ready to address that gap and use Delphi XE2 to do so.

Here's hoping Delphi XE2 / FireMonkey gains some widespread adoption and ushers in an age of resurgence in Delphi software development!

Java JDK 7 / JRE 7 Final Released : Java Runtime Environment and Development Kit, Five Years in the Making!

Java 7 / JRE 7 (Java Runtime Environment 7) and Java Final Released

Java 7.0 : A Five-Year Wait, Finally Released

Seriously folks, it has been a whopping 5 years since work was started on the successor to Java 6. In that time, Sun Microsystems was purchased by Oracle Corporation, and so much has changed! Think about where the web was 5 years ago and where devices were: you didn't have Android or iPad presence to consider and HTML5 was also nowhere to be found, Google's Chrome Browser was also not around. But, while Java sat in an apparent state of near-dormancy (with regards to major development milestones), the rest of the web changed and JavaScript/HTML5 came into existence and all those devices, browsers, and other technology really took off.

I am one of those developers that kept looking at Java every year or so and contemplating using Java as a software development language / framework, but ultimately choosing each time to NOT go with Java for the simple reason that it looked stagnant and lacking. There were so many missing features that I considered a requirement for any software development language, and the promise of those features coming in Java 7 just continually kept taking more and more time to become reality. I gave up on Java 7 repeatedly with my last major evaluation of the state of Java 7 taking place in mid-2010, where I again passed on it in favor of alternative technologies.

But, yesterday without much fanfare, Oracle finally released the "final" (or should I say "initial") version of Java JRE 7. And, it is not too surprising they didn't make a big deal out of the release, as (from what I observed over the past few years), the only way they could even achieve this 5-year-release milestone was by peeling away features that had been planned for this release (and which are now postponed or will never happen). But, regardless, the JRE 7 / Java 7 release is now available.

What's New in Java 7 / JRE 7 / JDK 7 / Java SE 7

OK, the first thing I can not stand about Java is all this "JRE, SE, EE, JDK, etc." stuff. For god sakes... why so many names? There is the Java Runtime Environment (JRE 7) and the Java Development Kit (JDK 7) and then the confusing "SE" nomenclature (Java Platform, Standard Edition) and "EE" (Java Platform Enterprise Edition), and all the other junk like Javabeans and Enterprise JavaBeans and on and on.

This alone says a lot about what I consider Java: a mess. As much as I am no huge fan of Microsoft's bloated DotNet framework, I consider it a huge improvement (and much simpler to deploy applications based on) over the Java counterpart(s).

Summary of New Features in Java 7

At a very high level, Java 7 includes new features like: some changes to the Java language that should lead to developer productivity improvements; a new Filesystem API and support for asynchronous I/O and even built-in zip-file handling (Java I/O features the new NIO 2, Non-blocking I/O 2); some updates to portions of the framework for multicore performance (like in the fork/join area); improved support for dynamic and script languages; security updates and improvements; and other stuff

Details of New Features in Java 7

I now come back to that whole JRE, JDK, SE, etc. stuff... as I have to tackle the What's New in Java 7 by each of these sub-categories. And, then comes how to even quickly assess the new features in Java 7 when Oracle puts out a page that clearly targets "techno geeks" with terms like RFE, without them even taking time on their web-page to define or explain what an "RFE" is!

So, let me start there, an RFE in the context of Java improvements/bug-fixes/feature-requests is:

Java RFE: Request For Enhancement — i.e., a formal request to Sun (well, Oracle now) to enhance the Java language or the related libraries.

People have been submitting these RFE's for years, and if they are lucky, and enough people really push for something, the RFE may be implemented in a future release of the Java JRE/JDK, etc.  In this case, the Java 7 release includes enhancements that address some rather significant and long-standing RFEs (including a few that kept me from ever commiting to developing any substantial application using Java).

What's New in Java SE 7 and Java JDK 7

Changes to Java SE 7 include changes to the Java language, the definition of the Java Virtual Machine (JVM), or the Java SE API Specification. Changes to JDK 7 includes changes to javac, to HotSpot (and related tools), and to the implementation of the Java SE 7 API. Wow, I already feel like I am going in circles where the JDK 7 changes reference the SE 7 changes and back-n-forth. Ughhhh.

Well, regardless of the absolute mess (in my view) that Java is in, in terms of naming and functional areas of the technology, here are some of the more substantial items in this release (I focus first on items most important to me as a software-developer looking at Java for creating custom software applications):

• Java Programming Language Enhancements : What could be more important to Software Developers? (well, a lot actually,... a great language alone means little without a framework that supports common needs like easy file-I/O, which thankfully is also addressed in Java 7). There are some nice improvements to the programming language worth mentioning here,including:
• Strings in switch Statements : I wish all languages could standardize on supporting this; to me it only makes sense to be able to base a case (switch) statement's execution-branch evaluations on anything, String class or otherwise. Nice!
• Type Inference for Generic Instance Creation : I find this new language feature to be quite nice, and it is representative of what I would expect in most "modern" languages. You can now replace the type arguments required to invoke the constructor of a generic class with an empty set of type parameters (<>) as long as the compiler can infer the type arguments from the context. This pair of angle brackets is informally called the diamond.
• The try-with-resources Statement : I find this new language feature to be quite interesting. There are many times where I can see this making a lot of sense, especially with database connections and the like. The try-with-resources statement is a try statement that declares one or more resources. A resource is an object that must be closed after the program is finished with it. The try-with-resources statement ensures that each resource is closed at the end of the statement. Any object that implements the new java.lang.AutoCloseable interface or the java.io.Closeable interface can be used as a resource. The classes java.io.InputStream, OutputStream, Reader, Writer, java.sql.Connection, Statement, and ResultSet have been retrofitted to implement the AutoCloseable interface and can all be used as resources in a try-with-resources statement.
• Catching Multiple Exception Types and Rethrowing Exceptions with Improved Type Checking : This complements the previous feature nicely, and this should be an improvement over various nested-exception-handling alternatives. A single catch block can now handle more than one type of exception. In addition, the compiler performs more precise analysis of rethrown exceptions than earlier releases of Java SE. This enables you to specify more specific exception types in the throws clause of a method declaration.
• Binary Literals : the integral types (byte, short, int, and long) can also be expressed using the binary number system by adding the prefix "0b" or "0B" to the number. Not a huge change, but nifty.
• Underscores in Numeric Literals : I consider this a "nice to have" more than anything. You can now use any number of underscore characters (_) anywhere between digits in a numerical literal to separate groups of digits in numeric literals.
• Improved Compiler Warnings and Errors When Using Non-Reifiable Formal Parameters with Varargs Methods
• New Java Virtual Machine (JVM) Abilities in Java SE 7: Support for Non-Java Languages (via a new JVM instruction that simplifies the implementation of dynamically typed programming languages on the JVM), Garbage-First Collector (a server-style garbage collector that replaces the Concurrent Mark-Sweep Collector (CMS)), and various Java HotSpot Virtual Machine Performance Enhancements
  
• Enhancements in Java SE 7 JDBC : JDBC 4.1 introduces the ability to use a try-with-resources statement to automatically close resources of type Connection, ResultSet, and Statement and support for RowSet 1.1 (the introduction of the RowSetFactory interface and the RowSetProvider class, which enable you to create all types of row sets supported by your JDBC driver.)
• Enhancements in Java I/O: [this was an area that always prevented me from using Java for any "real" software development work] thankfully, this area got a fair amount of attention with Java I/O features being improved considerably with the new NIO 2, Non-blocking I/O 2. If you need to work with files (I always wondered: "who does not need to?"), this should make Java more usable for what I call "real" applications.
• Swing Enhancements in Java SE 7 : this includes new or improved items including the JLayer Class (a flexible and powerful decorator for Swing components), the Nimbus Look & Feel, support for some mixing of Heavyweight and Lightweight Components, support for Shaped and Translucent Windows (think, modern transparency and non-rectangular shapes), and a tab for Hue-Saturation-Luminance (HSL) Color Selection in JColorChooser Class.
• Networking Enhancements in Java SE 7: a URLClassLoader.close method has been added.
• Java™ SE 7 Release Security Enhancements: a slew of new security related enhancements including a native Elliptic Curve Cryptography (ECC) provider; CertPath Algorithm Disabling (allowing weak cryptographic algorithms to be disabled), and updates to JSSE (SSL/TLS) to support TLS 1.1, TLS 1.2, Connection-sensitive trust management, Endpoint verification, TLS renegotiation, and Server Name Indication (SNI) for JSSE client.
• Concurrency Utilities Enhancements in Java SE 7: The fork/join framework (based on the ForkJoinPool class) is an implementation of the Executor interface designed to efficiently run a large number of tasks using a pool of worker threads.
• Client JRE Capabilities (RIA - Rich Internet Applications in particular) have been further improved to create what I call "modern" looking applications.
• Java 2D Enhancements in Java SE 7: a new XRender-based Java 2D rendering pipeline is supported for modern X11-based desktops, offering improved graphics performance; Support for OpenType/CFF Fonts, and Support for Linux Fonts(where libfontconfig is used to select fonts to use for the logical fonts for "unrecognized" Linux platforms)
• Plus other enhancements to: Java XML Technology (JAXP, JAXB, JAX-WS); Multithreaded Custom Class Loaders in Java SE 7 (potential deadlocks were eliminated for multithreaded, non-hierarchically delegating custom class loaders),

Some Other RFE's addressed in Java 7

The following is a list of some of the RFE's that were addressed in Java 7 that I considered noteworthy as I dug through the rather lengthy list of RFEs discussed in the release notes for Java 7 (there are quite a few more, but these are things I had interest in):

• (Tools) A succinct and concise error message is printed, as appropriate, when an error occurs during the Java launcher initialization.
• (Tools) The HtmlConverter tool is no longer part of the JDK 7 distribution
• (Tools) The javadoc tool has been updated to use style markup classes that can control the styles of the generated pages.
• (HotSpot) The default out-of-the-box heap size and shape parameters for the concurrent mark sweep collector (CMS) have been modified. The new settings take advantage of faster platforms [yeah, I'd say so in five years!] that have introduced since JDK 6 was released.
• (HotSpot) The GarbageFirst (G1) garbage collector is experimental in this release.
• (Plugin) 64-bit toolkit is now supported on 64-bit Windows platforms.
• (Plugin) Google Chrome is now supported by the DT Plugin (DT being deployment toolkit I believe).
• (API: AWT - Abstract Window Toolkit) All java.awt.Window objects (not frames or dialogs) are regular top-level windows on X11.
• (API: AWT) The mechanism used to detect whether the window manager supports a system tray has been changed (now uses the System Tray Protocol Specification)
• (API: AWT) Limited mixing of heavyweight (e.g. AWT) and lightweight (e.g. Swing) components is now officially supported.

More on What's New in Java7 / JRE7 / JDK7 / Java SE7

I hope I have given you a feel for what to expect with regards to new features in Java 7, especially features that may be interesting to you as a software developer. I MAY take Java seriously as a potential development language now, but... I need to dive into things like the file-I/O in more detail and see how it compares to what I am used to with Embarcadero Delphi (VCL Framework) or Microsoft DotNet Frameworks and development environments and languages.

Also, there were a LOT of big features that were highly anticipated (by me, and others) originally slated for Java 7 that did not make it into this release. Things like unified and robust date / datetime / time support. To me, I can hardly take any language/framework seriously if it does not provide native support for this type of thing. Sure, I can add on the the support I need (perhaps) with something like Joda / Jodatime or whatever, but please... why is something so basic (as a business software requirement) not included in the basics of the framework, language, types, etc? I find that really annoying, and another reason I avoid Java (or have avoided; time will tell if my avoidance is permanent).

 I really WANT to use Java for some things and want to be able to use Java for certain things — in particular I have some projects I wanted to implement using NetBeans platform components, which would sure imply Java as my primary development language to go with it ideally. Well, we shall see. If you want more information see this Java7 link to Oracle's page of Java 7 New Features and Release Notes and such.

Get ready to dive into all sorts of detail and read those lovely "RFE" entries!

VMware ESXi 5.0 New Features and vSphere 5.0 New Features

UPDATE: see my newer blog entry for latest (Sept-2012) VMware ESXi 5.1 info.

I recently posted a blog about the expected VMware ESXi 5.0 New Features, and since that posting just a couple months ago, VMware has released to the public the details of new features in VMware ESXi 5.0 and vSphere 5.0; so, this is a followup blog to cover the "official" new features in ESXi 5.x ...

New Features in VMware ESXi 5.0

Before getting into the new features in vSphere 5 / ESXi 5, I want to start with some helpful terminology, in case you are not familiar, as this will be mentioned a few times in the coming paragraphs as it relates to new significant features in the VMware products: ".VIB" files. You may have seen these VIBs before when you were upgrading VMware (e.g., as described in my how-to upgrade VMware ESXi 4.0 to 4.1 blog).

A .vib file is an ESXi / vSphere (or VMware) Installation Bundle (VIB file) that is essentially just a Debian (Linux) Package from what I have been able to deduce; and, not surprisingly, these .vib files may be referred to as "Software Package" files too.  These .vib files come into play as VMware (and partners / providers) package their solutions, drivers, CIM providers, and applications which are designed to extend the ESXi platform in some way.

Those .vib files are going to play an important role for the new Image Builder and Auto Deploy features, as any given ESXi Image is composed of VIB files (software packages) for 1) the core hypervisor, 2) Drivers, 3) CIM providers, and 4) other plug-in components. And, .vib files can include information about relations with other .vib files (e.g., ones they depend on and/or may conflict with).

ESXi 5.0 : Image Builder (NEW)

Image Builder is a new new set of command line utilities that allow administrators to create custom ESXi images that include 3rd party components required for specialized hardware (like drivers and CIM — Common Information Model &mdash providers). These utilities are used to create images suitable for different types of deployment, such as ISO-based installation, PXE-based installation (Preboot Execution Environment), and Auto Deploy (a new mechanism for provisioning ESXi hosts under vSphere 5.0)
Image Builder is designed as a Power Shell snap-in component and bundled with PowerCLI.

Image Builder is meant to help overcome limitations that exist where the Standard ESXi image (ISO), with its base providers and base drivers, does not have all the drivers or CIM providers for your specific hardware and/or does not otherwise include the vendor-specific plug-in components you require. Put another way, Image Builder will allow you to create and manager image profiles and build ESXi customized boot images (e.g., installable .ISO images or bundles ready for PXE installation). These images can then be placed in a "Depot" (a repository containing your image profiles and relevant VIBs), and these "Depots" can exist as a web-server-type-depot or a ZIP-file-encapsulated-depot .

Ultimately, ImageBuilder will allow you to then clone and modify an existing image profile, selecting the various VIBs you want to incorporate (be it drivers, ESXi VIB files, and/or OEM VIBs), and generate an ISO image (for a CDROM/DVDROM) and/or a PXE-bootable image. Whew! Got that? Well, this should at least give you an idea of what the new feature is for... and how it also goes along with the next new feature...

ESXi 5.0 / vSphere 5 : Auto Deploy (NEW)

Since I mentioned it in the prior paragraph, here is a bit more discussion about the new "Auto Deploy" construct. Quoted from the VMware "What's new in vSphere 5" PDF, "Auto Deploy is a new deployment and patching model for new vSphere hosts running the ESXi hypervisor. Deploy more vSphere hosts in minutes and update them more efficiently than ever before."

So what exactly does that mean?

Auto-Deploy is based on PXE-Boot (i.e., the ability to boot an ESXi host over the network), and works with Image Builder, vCenter Server, and Host Profiles.  It works like this: PXE boots the server, the ESXi image profile is loaded into host-memory via the Auto-Deploy Server, configuration information is applied (using Answer File / Host Profile  —  answer files being a new 5.0 construct), and the host is placed/connected in vCenter.

This has substantial advantages in that it requires no boot disk (on each connected host), and theoretically you can use it to quickly deploy a large number of ESXi hosts over the network and share a standard ESXi image across many hosts — the ESXi host-image has been effectively "de-coupled" from the physical server; secondarily this implies that you could recover a (failed) host without recovering the physical hardware and/or restoring from a backup [NOTE: an assumption here is that you have another server available with matching hardware].

This new (Auto Deploy) approach has a lot to offer!  This should allow you to setup a single boot-image that can be shared across hosts; and, with every reboot each host is starting up with a consistent image (keeping you servers all in-sync with regards to their ESXi setup).  This will likely replace a lot of custom scripting and other home-spun solutions currently in existence.  I can see this being a HUGE advantage when dealing with massive server farms.

With all "state information" being stored off the host and in vCenter Server (rules store; host configs saved in host profiles; custom host-settings saved in "answer files"), one has to wonder (well, I certainly wonder), whether this is putting "all your eggs in one basked" a bit? Also, this clearly implies one serious Enterprise-grade setup for your network and hardware (can you say, redundancy and fault-tolerance!?).  All steps of the Auto-Deploy (to me at least) certainly will require a rock-solid infrastructure: from 1) PXE Boot, to 2) the boot contacting the auto-deploy server, to 3) the auto-deploy rules-engine determining what image profile, host-profile (and cluster) is being dealt with, to 4) pushing the appropriate image to the host requesting it and applying the proper host profile, to 5) placing the host into a cluster... that all sounds a bit intensive to me.

Well, that is a summary of what this new "Auto Deploy" is all about... hope it at least gets your interest.  I have no way to really test this out much in my environment, as I do not have piles of servers sitting around to play with, etc.

ESXi 5.0 : Firewall (NEW)

Quoted from the VMware "What's new in ESXi 5" web page: "The ESXi 5.0 management interface is protected by a service-oriented and stateless firewall, which you can configure using the vSphere Client or at the command line with esxcli interfaces. A new firewall engine eliminates the use of iptables and rule sets define port rules for each service. For remote hosts, you can specify the IP addresses or range of IP addresses that are allowed to access each service."

I guess this is something I should get excited about. I did always find it (a bit) concerning that ESXi had no built-in firewalling in previous versions of the product. Has it ever caused me an issue? Well, no... but, I also have another layer of hardware firewalls around my servers that has (so far) done the job just fine. But, more security is generally a good thing, and presuming it does not impact performance (which, it seems is very unlikely), I see no reason why I would not use it.

New (Version 8) Virtual-Machine format with 3D (Windows Aero) and USB 3.0 Support : Great News for us Software Developer Types!

Since I use VMware ESXi for a lot of software-development-related tasks (including testing software under various operation systems for consistency, etc), I was thrilled to see support for 3D graphics (though, non-hardware-accelerated support) for Windows Aero (this is going to be quite handy for my Windows-7 x32/x64 testing).

And, I am definitely looking forward to the USB 3.0 device support. There are some limitations with this support, but I still hope to take advantage of any increased throughput to my external USB3 devices.  Again, quoted from VMware's "What's new in ESXi 5.0" web page: "ESXi 5.0 features support for USB 3.0 devices in virtual machines with Linux guest operating systems. USB 3.0 devices attached to the client computer running the vSphere Web Client or the vSphere Client can be connected to a virtual machine and accessed within it. USB 3.0 devices connected to the ESXi host are not supported at this time."

So, the wording of that paragraph could certainly be better.  What it sounds like is that VMware was unable to implement support for USB 3.0 devices under Microsoft Windows based Guest Virtual Machines at this time.  And, although the Linux-guest VMs can connect to a USB 3 device, that "connection" is essentially made over just a "pipe" of sorts that is passing data flowing to a local USB 3 connection (local to a machine running the vSphere client) over the network to our ESXi-hosted-VM?  So, to say that another way... it looks like we are going to be able to connect USB 3 devices, to Linux-guest-VMs only, and even then we will be passing that USB3 traffic over our network?  Wow, that surely diminishes my initial excitement with this feature, but at least I can still "attach" a USB3 device to an ESXi-hosted VM (not quite the same as plugging that device into my server, directly, though!)

I would expect this USB 3.0 support to improve with ESXi 5.1 or such.  I can understand why this approach would have been taken, as it would be much simpler to implement.  The fact is, if a local machine (running vSphere client) has access to a USB 3.0 device, we already have the two things we need: a network connection to ESXi, and a computer and OS that supports USB 3.0 devices and can "talk" to them... so, what easier way to implement some type of support in ESXi (VMs) than to pipe data across the network to those VMs.  But, if that was the reasoning, I do not fully "get" why only Linux VMs would support this.  Maybe I am just confused.  I need to do some testing and see what I run into.

Other new virtual-hardware features:

• 32-way virtual SMP
• 1TB virtual machine RAM
• UEFI virtual BIOS. Virtual machines running on ESXi 5.0 can boot from and use the Unified Extended Firmware Interface (UEFI).
• Smart card reader support for virtual machines (implemented *somewhat* like USB3 support in that this refers to smart card readers attached to the client computer running the vSphere Web Client or the vSphere Client); those smart cards can be connected to one or more virtual machines running under ESXi. And, the virtual machine remote console (in the vSphere client products) support connecting smart card readers to
  multiple VMs, making them useful for smart card authentication to VMs.
• Apple Mac OS X Server guest OS support — but, before you get TOO excited, this new vSphere 5.0 support for the Apple Mac OS X Server 10.6 (“Snow Leopard”) as a guest operating system is restricted to running only on certain Apple Xserve model hardware. I guess if you are "an Apple shop", this will be a neat feature.  Otherwise, it is a bit lack-luster from where I sit... I would have very much liked to run OSX 10.6 or 10.7 and such inside a hosted VM so I could use it for testing web-sites under Safari and other things.  Oh well.
  

Other New and Enhanced ESXi / vSphere Features

In no particular order...

• Swap to SSD. (I am very interested in this, as I hope to make use of my Intel SSDs that have done so well already on my desktops and existing ESXi 4.1 server). vSphere 5.0 provides new SSD handling and optimization whereby the VMkernel automatically recognizes and tags SSD devices (local to ESXi or available on the network), and can apparently use this information to allow the ESXi swap to extend to those SSD devices and enable memory-over-commitment while minimizing the performance impact).
  
• Support for SNMP v.2 with full monitoring for all hardware on the host.
• Secure Syslog (system message logging) enhancements.
• Enhanced Unified CLI Framework, including the ability to use the esxcli framework both remotely as part of vSphere CLI and locally on the ESXi Shell (formerly Tech Support Mode).
• Profile-Driven Storage, Storage I/O control, Network I/O Control, and Distributed Switch for additional Enterprise-Level performance and control features.
• And, oh yeah... you may have noticed the mention (throughout this blog) of the vSphere Web Client... that too is new, and as the name implies, it is a browser-based implementation of the vSphere Client... and, it is (at this time) a limited subset of what the full vSphere Windows UI / Client contains.  I expect this will eventually evolve into the full-fledged vSphere client over time, thus making the management client available on all clients (and OS's), via modern web browsers.

The fact is, there is a LOT of new features in this latest release of ESXi 5.x and vSphere 5.x that are worth checking out.  If you need more details, I suggest going to VMware's website and checking out all these products, whitepapers, announcements, and such.