Nvidia CUDA Toolkit 3.2 - more Fermi optimizations

The latest Nvidia "Fermi" GPUs (Graphical Processing Units) are available in a wide range of powerful new Nvidia Graphics Cards. If you have been waiting for a good reason to upgrade your graphics card, Nvidia has certainly provided some great reasons to upgrade with their latest CUDA-capable Nvidia cards that are just amazing!

For over a year, I was patiently waiting for the new Fermi architecture to hit the shelves so that I could get the latest CUDA parallel processing power at an incredible performance-per-watt and price-per-performance level. I have been looking forward to writing some parallel-processing-optimized code, and these cards were exactly what I needed. I ended up buying a Quadro 600 to start with, since it has 96 processing cores and only uses 40 watts of power! So far, I have been quite impressed by this card's abilities, quiet operation, and price (well under $200). I can now perform "real time" graphical operations that were impossible before. It is like having a 5 or 10 year old multi-million-dollar supercomputer on my desktop for under $200.


Personal supercomputing for the masses!
Nvidia (NASDAQ:NVDA) has moved to a modern 40nm architecture for these new GPUs, which has allowed them to be much more power-efficient while cranking out tons of graphics horsepower for gaming and/or professional applications that make use of their stream-processors (aka, "CUDA cores") on the graphics card for high-performance computing (HPC) via massively-parallel-processed algorithms.

CUDA is NVIDIA’s parallel computing architecture that enables dramatic increases in computing performance by harnessing the power of the GPU (graphics processing unit) for applications including image and video processing, computational biology and chemistry, fluid dynamics simulation, CT image reconstruction, seismic analysis, ray tracing, and much more.

Get an NVidia Fermi-based Graphics Card
First, get hold of a new Fermi-based Nvidia CUDA Graphics card  to develop and run your new CUDA applications on.

Now you can start putting some new CUDA abilities to work using the latest Nvidia CUDA Toolkit 3.2 release that has some features specific to the new Fermi cards and architecture that you may want to check into...


Nvidia CUDA Toolkit 3.2 Release Highlights

New and Improved CUDA Libraries

• CUBLAS performance improved 50% to 300% on Fermi architecture GPUs, for matrix multiplication of all datatypes and transpose variations
• CUFFT performance tuned for radix-3, -5, and -7 transform sizes on Fermi architecture GPUs, now 2x to 10x faster than MKL
• New CUSPARSE library of GPU-accelerated sparse matrix routines for sparse/sparse and dense/sparse operations delivers 5x to 30x faster performance than MKL
• New CURAND library of GPU-accelerated random number generation (RNG) routines, supporting Sobol quasi-random and XORWOW pseudo-random routines at 10x to 20x faster than similar routines in MKL
• H.264 encode/decode libraries now included in the CUDA Toolkit

CUDA Driver & CUDA C Runtime

• Support for new 6GB Quadro and Tesla products 
• New support for enabling high performance Tesla Compute Cluster (TCC) mode on Tesla GPUs in Windows desktop workstations

Development Tools

• Multi-GPU debugging support for both cuda-gdb and Parallel Nsight 
• Expanded cuda-memcheck support for all Fermi architecture GPUs 
• NVCC support for Intel C Compiler (ICC) v11.1 on 64-bit Linux distros 
• Support for debugging GPUs with more than 4GB device memory 

Miscellaneous

• Support for memory management using malloc() and free() in CUDA C compute kernels 
• New NVIDIA System Management Interface (nvidia-smi) support for reporting % GPU busy, and several GPU performance counters

New GPU Computing SDK Code Samples

• Several code samples demonstrating how to use the new CURAND library, including MonteCarloCURAND, EstimatePiInlineP, EstimatePiInlineQ, EstimatePiP, EstimatePiQ, SingleAsianOptionP, and randomFog 
• Conjugate Gradient Solver, demonstrating the use of CUBLAS and CUSPARSE in the same application 
• Function Pointers, a sample that shows how to use function pointers to implement the Sobel Edge Detection filter for 8-bit monochrome images 
• Interval Computing, demonstrating the use of interval arithmetic operators using C++ templates and recursion 
• Simple Printf, demonstrating best practices for using both printf and cuprintf in compute kernels 
• Bilateral Filter, an edge-preserving non-linear smoothing filter for image recovery and denoising implemented in CUDA C with OpenGL rendering 
• SLI with Direct3D Texture, a simple example demonstrating the use of SLI and Direct3D interoperability with CUDA C 
• cudaEncode, showing how to use the NVIDIA H.264 Encoding Library using YUV frames as input 
• Vflocking Direct3D/CUDA, which simulates and visualizes the flocking behavior of birds in flight 
• simpleSurfaceWrite, demonstrating how CUDA kernels can write to 2D surfaces on Fermi GPUs

Financial Opportunities :
Nvidia (NASDAQ:NVDA) stock?

Since this blog also focuses on stock-market and investing opportunities, I have to point out that in my August 13th, 2010 blog entry about Nvidia Toolkit 3.1 news, I contemplated whether the new Nvidia Fermi cards were going to drive substantial sales/revenue-gains and associated profit-gains for Nvidia corporation.

When I wrote that blog entry in August, Nvidia stock was $9.39, and today it is $14.77 as I write this article. If you jumped in on this one, you have already made 57% in a mere 4 months! (update: May-2011; NVDA at $19.00+) The current trend-lines on the stock look good, as it is staying ahead of its moving-average trend-lines on a technical basis, so it may well have some decent upside remaining. Plus, we are rather early in the Fermi-chip-based GPU series from Nvidia. 50+% return on your NVDA stock may have you wanting to take some profits. The choice is yours... this stock has a long-history of being rather volatile, and it will likely have some up/down cycles during its future. I plan to maintain at least some of my position in NVDA as I still think they have the technology to beat when it comes to GPU-computing and supercomputing. Intel's "Sandy Bridge" products coming out in Q1-2011 may have a slight impact on Nvidia (since the new Intel CPUs will include an integrated and allegedly rather capable GPU onboard, which will perhaps suffice for mainstream users).


WHO is going to use these cards?
The thing that I see happening with these discrete graphics cards like the Fermi-based CUDA-capable Nvidia cards is simple: if you are a business and you want to compete, you best learn how to leverage the power of these cards. Period.

Wall Street already knows this (and, I do not mean in the price of NVDA stock),... they are using this technology to perform lightning-fast calculations for algorithmic trading, options pricing, and much more. ANY application that can be significantly enhanced (i.e., made faster and more robust) through parallel processing will be made so by the companies that are leading in any field. They WILL use these GPUs from Nvidia to accomplish that feat.

I am not investing in Nvidia for the fact that home "gamers" and the like enjoy their super-potent "GeForce" cards... I am in this because businesses are going to use TONS of these cards/GPUs in their "desktop supercomputers" for analyzing all sorts of things throughout their domain. Mark my words: companies that miss this opportunity (to leverage CUDA and parallel processing) are going to find themselves looking like Blockbuster as compared to Netflix now.

The margins on Nvidia's "Quadro" business-oriented line of cards is likely higher than that on the consumer "GeForce" line, and certainly their Tesla dedicated supercomputing-desktop devices are going to be money makers as businesses figure out how to use these (i.e., find talented developers to help them write some seriously cool parallel-enabled algorithms and applications software). This may take a while yet, but I would say that within 3-5 years, MOST serious business applications will make some use of CUDA and/or GPUs for heavy analytical processing.

Bottom line: NVIDIA HAS SOME AWESOME GRAPHICS CARDS TO CONSIDER, and some nicely updated tools to go with them!

Microsoft Certification Tests : Cheap Price Overseas, but not in USA!

I was just researching the cost of taking a Microsoft Certification course for SQL-Server database development, and I was poking around the Prometric web site when I stumbled upon the fact that there are price-differences between what technology professionals in the USA must pay for the exact same Microsoft certification test that a resident of India or China can take (note: they get it for much less)! I found this VERY concerning, Microsoft! In fact, it really has me re-thinking the whole certification thing in general.
 
Perhaps I am just in a serious anti-import and anti-outsource mood after watching a PBS program about the collage of the U.S. furniture industry and manufacturing in general (whacked by cheap imports), and the gloomy unemployment picture in the USA, all while we see imports rising, outsourcing rising, and everything generally stacked against the US worker these days.

How are we Americans supposed to "compete" on a level playing field when our own USA corporations - in this case Microsoft, with Prometric as their co-conspirator - increase the cost-basis of our own education/certification as compared to our foreign "competition"?

In the case of cheap Microsoft MS certification (for those in foreign countries like China and India), I can not help thinking that this is a direct assault upon the same American workers that helped Microsoft achieve their corporate software dominance during the past couple decades. We get "certified" and help sell and implement solutions that use Microsoft software, but yet we have to pay more for the privileged of being a USA Microsoft Certified developer than our foreign counterparts that compete for our jobs with global-outsourcing gigs and the like. Lovely.

More details: cheap MS Certification test prices (if you live in the right country)...

So, Microsoft makes it cheaper for residents of foreign countries to get Certified in MS products than USA citizens, so as to further enable them to have an advantage over USA labor. I.e., reduce their total costs for achieving the same (certified) status and make it even easier to under-cut onshore (USA) IT talent.

Microsoft uses a company called ProMetric to administer their MS technolgy-certification tests *worldwide*. And, you can access their website and choose what country you are in and the see the rates (in US Dollars) for testing in each country.

For my own price-comparison, I looked at the cost of Microsoft Certification Exam 70-433: TS: Microsoft SQL Server 2008, Database Development...

LOWER EXAM PRICES OVERSEASHere is the summary of Exam fees / price charged by country for the SAME TESTING (with all pricing being in US Dollars):

• USA: $125 
• India: $50 
• China: $50

And, as this quoted information from Prometric's web-site makes clear, you have to love how you (as a USA citizen) can not even get the advantage of lower rates by testing in a foreign country while traveling, should you ever have such occasion to do so:

"In order to sit for a Microsoft Certification exam at a Prometric testing center in India, China, or Pakistan, candidates must be a legitimate resident of that country. If Candidates ARE a legitimate resident of that country, note the following: 

Effective September 15, 2010, Prometric testing centers in these three countries are required to confirm and record that each Microsoft Certification candidate has shown documented proof that they are a legitimate resident of that specific country. To verify country of residence status, each candidate is required to bring two valid and officially recognized forms of identification (ID)—one with a photo and an official residence address, and both with a signature. Students in Pakistan may use Student IDs as an official form of identification. 

NOTE: Non-eligible candidates will be turned away at the testing center if the candidate is unable to show proof of residency."

YOU MUST BE JOKING, Microsoft!
That "residency" requirement says it all... there is no chance of arguing that somehow the price-difference with the testing has to do with "costs" of administering the test in each country (like, perhaps, lower office-space costs, or similar), since if that was the case, you would let ANYONE take the test in the foreign country and realize the lower-cost/overhead advantages by way of a lower total test-price to the individual taking the test. This is all about fleecing the American worker in a way quite similar to how American drug-makers sell their drugs to foreign governments cheaper than they will sell drugs to American citizens.... really, how is this any different Microsoft?

I (semi) understand the argument for discounting software in such countries as China, India, and other "developing" countries (since they notoriously just STEAL the software otherwise and pay little to no regard to Intellectual-Property law), but WHY discount an electronically-administered TEST there for certification-achievement?

The only potential argument I can come up with how to "justify" the low-price MS Certification tests in these foreign countries (from a MS standpoint), is to perhaps rationalize that: if you, Microsoft - Nasdaq:MSFT, have your pool of "MS certified" technical people in each country, those same countries are likely to use more of your software,...but, the more I think about that argument, the more I would have to respond "Microsoft, you must be joking!"

 
In the mean time, we software developers, programmers, database administrators, and related Microsoft technologists, are stuck essentially subsidizing the very same overseas information technologists that are going to try their hardest to outsource OUR jobs as we continue to import all THEIR goods and services -- the same goods and services made even more price-competitive thanks to Microsoft/Prometric giving our competition a discount on the same certification test that we need to take to maintain our IT credentials and supposedly stay "competitive"!

This is just another sick example of an economy that is rigged against us. Thank you Microsoft! (NOT)

Education vs. Income : China proves Bernanke wrong

If you find the current economic situation in America disheartening, and you want to understand "why" things are the way the are with regards to high unemployment and the massive disparity in wealth between the upper few and the rest, do not listen to Federal Reserve chairman Ben Bernanke: he is out of touch with reality! He may be a very smart guy, but he seems to be missing some blatant truths about what is going on at a macro-level in the USA and worldwide.

One of Bernanke's favorite "reasons" for the ills of this economy is: education. I watched him on a TV interview last week where he cited educational differences as one of the core determining factors between the have and have-not groups. I personally beg to differ Mr. Bernanke, and a morning story on CCTV (China Central Television) backed my position completely...

When it comes to Education vs. Income, it is just what I keep saying: a standard supply/demand curve sets salaries and earnings; that is all, nothing more. Education used to place you in a supply-constrained-group; not anymore.

China proves Bernanke wrong...China just proved it. The college-educated Chinese citizens were, just a couple years ago, on average making 4 times what the blue-collar factory workers were making in China. But, China is now experiencing an abundance of persons with degrees in accounting, engineering, technology, etc... and, those same college-educated persons are now making on average only 2 times the blue collar wages. The CCTV news show discussed how it is purely a matter of supply/demand for both sectors (white/blue-collar) that is forcing this narrowing in wages. 

China currently has an increased need for "low skill" labor, which has pushed the wages for the low end up while the college-educated job rates slow, stagnate, and decrease relative to the other group.


OK, Bernanke... how is the USA any different?There is a global glut of educated workers. As such, global wage-equalization is underway.

Makes you want to run out and get an MBA, doesn't it? NOT.

This premise that Chairman Bernanke makes, that lack of education is limiting incomes, is simply absurd in today's global economy. Sure, education *helps*, but it will not help beyond the point where supply exceeds demand for any one group of skilled labor - including college-educated labor.

SUPER-MACRO-EXAMPLE

The fact is, if EVERYONE on the planet had a college degree, and everyone had similar intelligence to back that degree, there would still be a need for "unskilled" labor in factories and in less desirable positions. And, who is going to serve up the Starbucks coffee in the morning... you got it: a college-educated person. And, pay differences are going to emerge like always, with stratification that does not reward education, but rather rewards being lucky enough to get one of the better jobs. 

There will still be the group on Wall Street that rewards themselves handsomely for doing nothing more than moving money around (electronically) and taking a "cut" and calling themselves brilliant investment bankers, private equity managers, and hedge-fund barons. It will have nothing to do with their education... just their position in society that allows them to siphon off huge amounts of cash from the productivity of everyone else in society.

Oh, wait... I just described the current economy in America!

What is REALLY wrong with the economy?

Bottom line: Bernanke needs to wake up and see the real issues of what is going on with the world economy and why this recession is now a supposed "jobless recovery". It has little to do with education. I, and many other people, know people with exemplary employment records and fantastic education credentials (including advanced degrees) that are unable to get work. 60-Minutes did a story where they featured a large roomful (100s) of the unemployed persons in Silicon Valley that were, predominantly over 40 and when asked what education-levels they had obtained, the majority of hands in the room were raised at "Masters" level and a large percentage were still raised at "doctorate/PhD" level.

Bernanke: how can we expect you to HELP this economy when you clearly do not understand the macroeconomic forces at play? Or, are you fully aware and just want simple excuses for the underlying mess that will never be addressed?

That underlying mess I refer to is an economy where:

• large numbers of college-educated persons can not find employment commensurate with their training or skills; 
• a *few* at the top of the economic food-chain, or the "top" as they have defined it to be, sit back funneling all the productivity out of this country via their instantaneous intraday flash-trading on Wall Street, whether for the banks they work at or the hedge funds and private equity firms and so forth; 
• there is "no inflation", but yet the volatile food-and-energy group, and any other thing that one truly *needs*, only goes up in price, and goes up faster than "inflation"; commodity prices are surging worldwide, yet there is "no inflation"... sure!; 
• there is no way for people to get any return on savings, though we always here how the USA got into this if from "lack of savings". Great, so now anyone who saves can look forward to .1% (yes, a tenth of a percent) or thereabouts on a "savings account" while large banks hit the same people up for 15+% on their credit-card interest rates. That is a 15,000% difference between what "savings" earn and what revolving-debt costs; need I say more? 
• it is nearly impossible to find ANY USA-MADE ITEMS AT STORES anymore. Period. I search to no avail. How am I to "stimulate the economy" (in the USA) if I can not find a way to spend my money on an item that would employ US workers? It gets worse every day. 
• Corporate America is moving jobs out of here as fast as they can... whether to dodge environmental code in the USA or to avoid the cost of benefits here or any other potential liability of having workers in the USA, they are rushing to China and other low-cost suppliers because our laws make it easy for them to do so. 
• IP (Intellectual Property) rights are only enforced when violations affect the largest of the large companies; aside from that, anyone starting a business these days will likely see their work stolen, cloned, or otherwise reproduced within a few months of product-launch, and there will be no repercussions. You will create a new product only to see a cheap Chinese knock-off on the market in no time. So, starting a product/business in the USA becomes even more foolish, and thus jobs do not stay here. 
• Tax-code that favors the highest-wealth people and our government that is "owned" by those people: I am sick of hearing things like "the uncertainty over taxes is preventing corporations from hiring"... GIVE ME A BREAK. There is no uncertainty! It is certain that, at worst case, taxes will revert to pre-Bush-tax-cut-levels... and, anything else is just a better case. How is a 2yr extension of a tax-cut going to lead to anything beyond another supposed "uncertainty" at the end of THAT 2yr period? Ludicrous argument! 
• And, we borrow money by the Trillions to pay for the implementation and materialization of the list I just enumerated!

Conclusion

I could continue enumerating all the (real) issues with the economy, but it will make no difference unless some people with influence start standing up to fix this mess.  And, fixing it is not just extending a tax-cut that favors the top one or two percent of the population.  Fixing it requires some serious change.

If we do not start creating jobs to provide educated workers with opportunity in-line with their skills, things are only going to get worse.  And, when China is already showing signs of a glut of educated workers, get ready for further wage-pressures here in the USA and even more unemployment among those educated persons that Bernanke seems to think do not exist in large enough quantities already.