Saturday, December 14, 2013

MSIE 11 - isnt

I got pretty upset when I was trying to figure out how to identify IE 11 from other browsers when trying to get a simple sound playing function to work on all browsers.
It turns out IE has decided in version 11 to lie about itself.
The userAgent string for IE11 no longer has the string "MSIE" in it - which has been there since it began.
Luckily, I did find a solution here.
But just because the magic of regular expressions comes to save the day doesn't excuse Microsoft for once again throwing yet another monkey-wrench into programming by lying.
The userAgent string has the express purpose of identifying to code what browser and version is running.  If you are viewing this on IE11 (which I am writing it on) and you select the help\about menu option, you will see it says "Internet Explorer 11" followed by a precise build number.
If you print the navigator.userAgent string you will find:

Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; rv:11.0) like Gecko

Now where do you see either "Internet Explorer/IE" or "version 11" in that string?  Well you can tell its MSIE because it says "Trident" and you can tell it's version 11 because it says "7.0" - simple right?
Most likely this change was decided upon because Microsoft has been trying to better emulate Firefox and this userAgent string is so much like a Firefox userAgent string as to fool existing code.
But IE is not Firefox and I know for a fact that there are still lots of differences between IE11 and the latest Firefix browser.
It is the short-cut to save time so that a million other developers, like yours truly, can spend hours searching for or inventing a solution.  It's the kind of lazy logic that makes the world just a little bit worse.
Thank you Microsoft-bozo, whoever you are, that made this LAME decision.


PS: Ah Microsoft receives the righteous consequence of their stupidity:
Google is not happy with IE11!

The browser we detected is unsupported and may result in unexpected behavior.
Please choose from our list of
supported browsers for the best experience.

Sunday, November 10, 2013


I just completed viewing a video of Eric P. Dollard entitled FOUR QUADRANT REPRESENTATION OF ELECTRICITY which I purchased from here.
I purchased this because I like Eric personally and I wanted to support him and because I didn't think I would have the time to get through the book.
Unfortunately, after finishing the video, I am sorely disappointed.
The only thing in the entire video that I didn't learn at BEEP and A school in the US Navy is a minor mathematical innovation that lets you express all four quadrants of an AC wave symmetrically.  (ie introducing a +j and -j coordinate space)
Unfortunately Dollard never develops the math or subject beyond basic AC theory applications.
Dollard is always fun to listen to because he has practical experience, deep theoretical and historical knowledge and tons of interesting experiences and anecdotes to relate.  This video is no exception, I was very entertained by his anecdotes and historical references.
The quality of the video production is atrocious and shows a typical short-sighted methodology to maximize profit and minimize work.
Eric is invited to talk at a Bedini-Lindeman conference and it is recorded with a very cheap camera and microphone system.  It is then run though a cheap (freeware most likely) video editing system and reduced down to minimum quality and file size (leaving numerous and annoying compression artifacts in the video) with still shots of Eric's diagrams taken from the projector image and photos overlayed onto the video at appropriate points.  This results in blurred images and diagrams you can't read, long sections of video where Dollard is looking off to the side at the projection screen where the images are and most likely were shot from (you can see the photo flashes going off during the talk).  All the video captures of Dollard pointing out parts of diagrams using a white light pen result in a pointer image that can barely be seen on the video.
Content is also disappointing because Eric tends to favor questions and interruptions from the audience pulling him off track.  Dollard actually openly says in the video that he had to leave 90% of the theory out of the talk - he just didn't have time to cover it. 
The Dollard lecture was clearly done before the book was completed and so Dollard speaks from scattered notes.

The real disappointment is that I paid $27 for this video that is below the quality found on most instructional videos free on u-tube which the selling webpage says is
"guaranteed to meet even your highest expectations.
P.P.S. Remember, you're not risking a single penny since it is 100% guaranteed!"


I shall be asking for my money back soon.  Perhaps I can talk them into giving me the book rather than my money back.  I can then review that for you as well here before you have to risk your cash to find out if it is worth the price.

In general, I have found disappointment to follow almost everything I have done with the Bedini-Lindeman group.  A consistent lack of professional care in production, organization and dissemination of knowledge along the lines of alternative energy topics and consistently overpriced.
I would not go so far as to say these guys are trying to rip people off.  However, these guys are trying to make a living off of the technology before it is ready and thus I think are forced into cutting corners that a professional would not.
How I would do a Dollard video:
  • Get Eric to get all his images and photos into a PowerPoint form.
  • Use a good PC recording tool such as SnagIt ($35) to sit down at a PC with a good mike and give his talk in a quiet room uninterrupted using his power point images without interruption.
  • Edit this down to remove mistakes and silence.
  • Create a 1080p version of the presentation.
  • Break the talk down into sections and put an index with timestamps at the front of the video.
  • THEN sell that!
I'll let you know how the book goes if I can get it exchanged for the video.
------ LATER -----
I asked for my $ back which had to go to Peter Lindeman, the man behind the sale of this lame video:

Sent: Sunday, November 10, 2013 10:54 AM
Subject: Request exchange on Four-Quadrant Representation of Electricity Video
I would like to propose an exchange of the video I recently purchased from for the book version.  (Rather than a refund)
Here is a review I have just published of the video which I think should explain my disappointment.
I would be happy to discuss with you ways to improve your products and might even be able to do services for you for a fee.
I had offered assistance to the Bedini crew years ago when I had published a review if one of their first conferences here.
Apparently I am not trusted due to actually being interested in helping them yet a relative stranger to them.
I have a friend who is a professional video editor and can do a much better job for you on videos as well – both recording and production.
I really do mean to be helpful to you but my consistent disappointments have caused most of my feedback to be negative.
Sandy Staab
Fellow Free Energy Advocate
To which he responded:
Dear Sanford,

Sorry you didn't like the film.  Your only recourse is to go back to Clickbank and get a refund.  There are no exchanges.  If you do get a refund for the film, please DON'T buy the book.  I assume you can find things to complain about in it, and since it covers the same material, you won't learn anything new from it anyway.

The film is a LIVE presentation to the conference audience.  The promotional materials make that quite plain.  The root camera file is cut in true HD and is over 38GB in size.  There were a few problems with the radio mics in the room, which our camera people were taking the feed from.  The file reduction to a downloadable size introduced some interlacing problems due to the MAC conversion to PC formats.  Beyond that, all of your criticisms represent your personal preference.

The purpose of the film is to make Eric's lecture, exactly as it happened, available to his students, followers and supporters.  We were specifically asked to leave it "uncut", which we did.

Your closing statement is quite telling.  The truth is, if you REALLY meant to be helpful, you would have found a way!


To which I responded:
On 11/10/2013 9:03 PM, Sanford Staab (GMail) wrote:  
Can you tell me how much of my purchase price goes to Eric Dollard?  I’d be happy to leave you with your money should I know that I at least helped Eric with my purchase.
To which he responded: 

The financial arrangements between Eric and his publisher are confidential.  Eric is pleased with the arrangement.  You should not expect to have this sort of information shared with you, as you are not a party to the contract.


To which I responded:
Ok.  I would just like to know if my purchase is helping Eric.  Sounds like it is.
After some searching I found a much better video (I think) on the very same subject (where Eric reads from his book/notes) along with the full text of Sir Edmond Whittaker's work for FREE:
It is sad that people who should know how to disseminate critical knowledge instead chose to make money off of the ignorance of others and pretend to disseminate that knowledge while at the same time hiding that knowledge.


Wednesday, October 30, 2013

Javascript OOpseseses

Yes, well I am confessing that I enjoy working with JavaScript for much the same reasons as Mr. Douglas Crockford does.  I was attempting to do a little OO coding, trying to basically specify an interface with a base class and inheriting from that class for various implementations of the interface.
This blog I found was quite helpful to me but still vague in how to properly use it.
So here is some short code I made up that convinced me of the right way to do inheritance.
<!DOCTYPE html>
  <title>Playing with OO javascript</title>
<script type="text/javascript">

// see
if (typeof Object.create !== 'function') {
    Object.create = function (o) {
        function F() {}
        F.prototype = o;
        return new F();
// Usage:newObject = Object.create(oldObject);

function createParentClassInstance() {
    var o = {};
    o.item1 = 'Parent says: hi.';
    o.item2 = 'I\'m the parent';
    return o;

function createChildClassInstance() {
    var o = Object.create(createParentClassInstance());
    o.item2 = 'I\'m the child';
    o.item3 = 'Child says hi.';
    return o;

function out(s) {
    document.body.innerHTML += s;

function PageLoad() {
    var parentInstance = createParentClassInstance();
    var childInstance1 = createChildClassInstance();
    out(parentInstance.item1 + '<br>');
    out(parentInstance.item2 + '<br>');
    out(parentInstance.item3 + '<br><br>');
    out(childInstance1.item1 + '<br>');
    out(childInstance1.item2 + '<br>');
    out(childInstance1.item3 + '<br><br>');
<body onload="PageLoad();">
Note how we hide the definition of the objects within a clearly defined constructor function.  Inheritance is controlled by the class factory so things become clear what they mean and its not easy to just throw objects together in random inheritance trees.
If would also seem that multiple inheritance could be done this way by simply cascading the instance creation functions to make a multi-generational inheritance structure.
I am hoping that Dr. Crockford will have a chance to comment on this blog and correct me if I am wrong here. 

Tuesday, July 9, 2013

Linear speaking fractally

Many years ago I wrote a line fractal application in C that uncovered a never-found bug in the GDI layer of WindowsNT.
It was a fun app to write and I have since redone it in C#.  I originally created an MSN Group to distribute it but as much of Microsoft's cloud support - feature one year, bug the next - it was dropped.

So here is a post just to expose it to anyone wanting to try it out:

Just download the .rar file (use WinRAR.exe to unpack) and run the .exe.
You will need some .Net runtime support - I believe it runs on .Net 3.0.
No certificate but I can vouch that its not a virus.

The idea of a line fractal is that of creating an alternative path between two points.  Then each sub-part of the alternative path is itself altered to use the alternative path scaled to fit the two new points.  After some recursion, you have a cool looking design.
Add some color, the option of drawing a line or not in the subpath and you have almost a work of art.


Wednesday, June 5, 2013

Poor Batch Scripts

I was whipping up some Batch scripts to rename .eml files for me so that the names included the date, subject, to and from fields to make it easier to organize and find my archived emails.
I have long used bat/cmd scripts to do stuff like this but this little project took me a full day with many breaks to relieve the frustration.
Knowing Steve Wood who built the cmd batch processor is cool but it still never helped me figure out this problem:

The variables in batch scripts can use a syntax which allows some string substitutions like:

%foovar:x=y%  which returns the value of the variable with all x strings replaced by y.
Its nice for simple cases but I found it impossible to substitute out * characters.
This is because doing:

%foovar:*=-% is specially interpreted to mean all of the variable contents up to whatever comes after the *.

I tried %foovar:^*=-%%foovar:"*"=-% and even %foovar:**=-% with no luck.


Wednesday, April 10, 2013

Ah the joys of working with Microsoft's SQL Server

The past 3 days have been largely unproductive because of the complexities arising from multiple versions of SQL Server and Visual Studio on my machine.  There is a known bug where if you installed VSExpress 2010 at sometime in the past and then try to install the SQL Server client tools you will have a dependency problem that prevents the SQL tool installation.
In my efforts to get this installed I pulled off all my SQL Servers and my Visual Studios but to no avail.  Even the hack found here failed me.
Today after many many hours of frustration I believe I have things back on track but man is my machine probably messed up getting to this state and I doubt I will remember all the steps and things I tried if I have to do this again.
Why is it that software for money is so complex?  I think Christ was right when he said that money is the root of all evil. 
The motivations to make money in software are contrary to the motivation to actually solve a problem.  New releases are needed to keep money flowing in - whether or not they are really needed.  And you can't charge someone for a bug-fix release - that is just too embarrassing to admit and to audacious to charge people for.  You need to package it in ways that maximizes the ROI.  You have to invent more fancy ways to use your solution, and you better not find a better solution that makes your old one obsolete - not unless you have competition forcing your hand.
SQL Server has historically been a pretty expensive product and not easy to get at the Microsoft Company Store.  Today I installed SQL Server Express 2012 - the competition has forced the great Microsoft to release its technology free.  MySQL and other free solutions are just too good to keep the fences up.
When working on Sidewalk, an early Microsoft website done before the days of ASP, a very clever developer on our team invented CQL (Conceptual Query Language) which consisted of an ODBC driver that accepted queries in a high-level language that was then translated into the optimal SQL query and cached.  Once this engine got loaded up it screamed and devs no longer needed to know much at all about the schema.
That dev was later hired by the SQL Server team, and I had high hopes to see CQL shipped as a revolutionary solution - but it never happened.
I doubt it could be due to a technical problem, as we were using it under real life loads in Sidewalk for years.
I think such a revolutionary solution would have put too many DB consultants out of business, including a whole division within Microsoft.
So we live with layer of crap over a pretty good basic DB engine and the whole world gets to download gigabyte plus sized install packages to deal with the immense complexities of products that have 20+ versions and packages out there.  Just look at this download page.
Code reuse - till the business model fails.

Sorry, I'm just burnt out on this stuff.

Thursday, February 14, 2013

SQL - the problem what should not be.

After spending about 5 hours trying to get a nested SQL query working I sit in frustration at how lame the language is and how even more lame the parsers are for the language.
They say the last 10% of any project is the hardest, and I agree.
We now have lame speech recognition but we don't have excellent SQL parsing.
What's worse, I know too much.  I know that this whole SQL thing was solved by the Sidewalk team at Microsoft back in 1997 by an ingenious developer who devised CQL (Conceptual Query Language) which was implemented using an ODBC driver on top of SQL Server.
We had a schema that was so big, it covered about 100 feet of wall-space around our offices when printed on huge 4x5 foot sheets of paper in 12pt font or so!  It became so difficult to generate efficient (even correct) SQL queries that CQL was devised.  It analyzed the schema and then, given a much more free-form language describing the problem in a table-less flat space would create the optimal SQL query for the CQL specification and cache it and return the results.  Parameters could be set up to substitute into the cached queries and, whalla, the whole SQL efficiency, complexity and parsing problem was gone.
That enterprising developer was hired by the SQL Server team.  Unfortunately, CQL never saw the light of day.  I suppose that it didn't make sense to solve a core problem like that.  There was just too much invested in SQL consulting and systems to bypass it.  I'm sure it came down to $.
Anyway, my specific problem is this:

-- phpMyAdmin SQL Dump
-- version 3.4.5
SET time_zone = "+00:00";
  `pageId` int(10) unsigned NOT NULL,
  `linkId` int(10) unsigned NOT NULL,
  `whenUsed` datetime NOT NULL,
  `whenRendered` datetime NOT NULL,
  KEY `pageId` (`pageId`),
  KEY `linkId` (`linkId`)
INSERT INTO `links` (`pageId`, `linkId`, `whenUsed`, `whenRendered`) VALUES
(1, 9, '2013-02-14 00:44:20', '2013-02-14 00:42:39'),
(1, 9, '2013-02-14 00:44:21', '2013-02-14 00:42:39'),
(1, 9, '2013-02-14 00:44:23', '2013-02-14 00:44:21'),
(1, 8, '2013-02-14 00:44:25', '2013-02-14 00:44:23'),
(1, 7, '2013-02-14 00:44:26', '2013-02-14 00:44:25'),
(1, 6, '2013-02-14 00:44:28', '2013-02-14 00:44:26'),
(1, 3, '2013-02-14 00:44:29', '2013-02-14 00:44:26'),
(1, 7, '2013-02-14 00:44:31', '2013-02-14 00:44:29'),
(1, 8, '2013-02-14 00:44:32', '2013-02-14 00:44:31'),
(1, 11, '2013-02-14 00:44:34', '2013-02-14 00:44:32');

Inner Query:
SELECT `linkId`, 
TIMEDIFF(`whenUsed`, `whenRendered`) AS 'URDiff', 
CONVERT('dateDiff' = 0, UNSIGNED) AS 'usedToday',
CONVERT('dateDiff' < 8 AND 'dateDiff' > 0, UNSIGNED) AS 'usedThisWeek',
CONVERT('dateDiff' < 31 AND 'dateDiff' > 7, UNSIGNED) AS 'usedThisMonth', 
CONVERT('dateDiff' < 365 AND 'dateDiff' > 30, UNSIGNED) AS 'usedThisYear'
FROM `links` 
WHERE MINUTE('URDiff') < 15 AND HOUR('URDiff') = 0
AND `pageId` = '1'
) AS T

Results of inner query:
| linkId | URDiff   | dateDiff | usedToday | usedThisWeek | usedThisMonth | usedThisYear |
|      9 | 00:01:41 |        0 |         1 |            0 |             0 |            0 |
|      9 | 00:01:42 |        0 |         1 |            0 |             0 |            0 |
|      9 | 00:00:02 |        0 |         1 |            0 |             0 |            0 |
|      8 | 00:00:02 |        0 |         1 |            0 |             0 |            0 |
|      7 | 00:00:01 |        0 |         1 |            0 |             0 |            0 |
|      6 | 00:00:02 |        0 |         1 |            0 |             0 |            0 |
|      3 | 00:00:03 |        0 |         1 |            0 |             0 |            0 |
|      7 | 00:00:02 |        0 |         1 |            0 |             0 |            0 |
|      8 | 00:00:01 |        0 |         1 |            0 |             0 |            0 |
|     11 | 00:00:02 |        0 |         1 |            0 |             0 |            0 |

+--------+----------+----------+-----------+--------------+---------------+--------------+ Outer Query:

SUM('T.usedToday') AS 'countToday',
SUM('T.usedThisWeek') AS 'countThisWeek',
SUM('T.usedThisMonth') AS 'countThisMonth',
SUM('T.usedThisYear') AS 'countThisYear'

SELECT `linkId`, 
TIMEDIFF(`whenUsed`, `whenRendered`) AS 'URDiff', 
CONVERT('dateDiff' = 0, UNSIGNED) AS 'usedToday',
CONVERT('dateDiff' < 8 AND 'dateDiff' > 0, UNSIGNED) AS 'usedThisWeek',
CONVERT('dateDiff' < 31 AND 'dateDiff' > 7, UNSIGNED) AS 'usedThisMonth', 
CONVERT('dateDiff' < 365 AND 'dateDiff' > 30, UNSIGNED) AS 'usedThisYear'
FROM `links` 
WHERE MINUTE('URDiff') < 15 AND HOUR('URDiff') = 0
AND `pageId` = '1'
) AS T

GROUP BY `linkId`        
Outer query result:+--------+------------+---------------+----------------+---------------+
| linkId | countToday | countThisWeek | countThisMonth | countThisYear |
|      3 |          0 |             0 |              0 |             0 |
|      6 |          0 |             0 |              0 |             0 |
|      7 |          0 |             0 |              0 |             0 |
|      8 |          0 |             0 |              0 |             0 |
|      9 |          0 |             0 |              0 |             0 |
|     11 |          0 |             0 |              0 |             0 |

Go figure?  I can clearly see from the inner results that there are 3 rows in the usedToday column that are 1 (and I have converted them to unsigned so they should SUM ok).  Yet the countToday for linkId 9 is 0 and not 3.  Why?

MySQL gives me some interesting warnings:
Warning (Code 1292): Truncated incorrect time value: 'URDiff'
Warning (Code 1292): Truncated incorrect time value: 'URDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'dateDiff'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedToday'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisWeek'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisMonth'
Warning (Code 1292): Truncated incorrect DOUBLE value: 'T.usedThisYear'

From what I have been able to find online about this warning, it appears to happen for some people who didn't us quotes on their names (which I use everywhere).  Now why would a parser give such a warning for quote usage?


My son correctly pointed out that I was quoting my column names (a no no I guess, with no help from the parser) and referencing aliases outside the GROUP BY, HAVING, and ORDER BY clauses (also no help from the parser).
The correct query becomes:
SUM(T.usedThisWeek)as countThisWeek,
SUM(T.usedThisMonth)as countThisMonth,
SUM(T.usedThisYear)as countThisYear
    SELECT `linkId`, 
    TIMEDIFF(`whenUsed`, `whenRendered`) as URDiff, 
    DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) as `dateDiff`,
    if(DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) = 0, 1, 0)as usedToday,
    if(DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) < 8 AND DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) > 0, 1, 0)as usedThisWeek,
    if(DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) < 31 AND DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) > 7, 1, 0) as usedThisMonth, 
    if(DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) < 365 AND DATEDIFF(`whenUsed`, UTC_TIMESTAMP()) > 30, 1, 0) as usedThisYear
    FROM `links` 
    WHERE MINUTE(TIMEDIFF(`whenUsed`, `whenRendered`)) < 15 AND HOUR(TIMEDIFF(`whenUsed`, `whenRendered`)) = 0
    AND `pageId` = 1
) as T
GROUP BY `linkId`    
With a cheery results of:

| linkId | ascountToday | countThisWeek | countThisMonth | countThisYear |
|      3 |            1 |             0 |              0 |             0 |
|      6 |            1 |             0 |              0 |             0 |
|      7 |            2 |             0 |              0 |             0 |
|      8 |            2 |             0 |              0 |             0 |
|      9 |            3 |             0 |              0 |             0 |
|     11 |            1 |             0 |              0 |             0 |

Friday, January 25, 2013

Where the real power of computers lies - raw text

There are two things that really drive computers:
  1. Conventions/Standards
  2. Raw Text
Almost every PC operation (with the exception of UI rendering of images, controls and sound) ultimately use simple raw text as the true container for information.  These simple text messages are sent across your machine and the Internet to transfer information.
Many times data is stored in a binary form and there are complex protocols for transferring and interpreting data, but really, when you unpack everything and see where it is ultimately used, it is generally just text.
Even the languages used to express programs consist of text.
This is because text embodies human-readable and interpretable information.  It is the ability of humans to read text that ultimately gives text its value and capacity to hold information.
This is a significant point which I think our modern gadget/image/sound driven world is forgetting as they design software.
Many times I see information (ultimately in the form of simple text of course), packaged into forms that are difficult to extract.  Even UI that presents information to you on say a web page or a PDF file often can prevent the extraction of the raw text for transfer to other places.
Just think about it.  Where would you be with out Cut/Copy and Paste?  The clipboard is probably the most-used feature of any GUI.  In the old command line days (still alive and well today) it was the ability to pipe text between commands that gave UNIX and DOS users the power to take disparate tools and combine them efficiently to perform complex tasks.
Can you imagine where we would be without piping and clipboards?
With standards like UTF-8 there just aren't any more excuses.  We can pass virtually all information in human-readable text and have the most efficient systems possible.
So here's my tip for the day - are you coding up tons of UI or interface code just to ultimately get some text across a wire or into another program or across to a user?  If so, you may be approaching your problem wrong.

Tuesday, January 8, 2013

Ideas on building software

Over my many years of programming time, I have often had ideas on how the tools should be made to make building software more efficient and pleasurable.
There have been lots of advances over the years but I, being the old fart that I am, still think very highly of Kernighan and Ritchie, and their first OS.
I long for the days of simplicity.
The creation of the IDE (Integrated Development Environment) is more of a boon to software companies wanting to rope developers into their pet APIs of choice.  After all, there is only so much a developer can learn so if you can keep him doing things your way you can sell him software forever.
There is so much creativity out there it's quite hard to corral these guys into a single paradigm, but for most it has been done.
I admit there is synergy in having a bunch of programmers all working on the same language and tools, I'm not against that, but I am against a single paradigm being boxed up and shoved down your throat - you want to work differently, then make your own tools.
The openness of a simple compiler, linker and make system I still like best.  The tools are individually comprehensible and manageable - especially if the input and output are mostly text files.
I wish they had gone further in developing the multi-phased approach to building software.
Here's a list of the parts I see as being nice to have separated and individually accessible:
A Pre-compile Phase Macro Package
The old C compiler had this built in which you could utilize by invoking a build so you could see the output for help with buggy macros.  But I think such an intermediate file is actually quite useful to have in the build process as a separate phase of the process.
It is incredibly useful to place things in one place and have other files build from those definitions in different ways. Maybe you have common constants you want to use across php, javascript, C and SQL.  Maybe you have some key core data that would really speed things up if it were hard-coded in each layer of your distributed design.  Keeping things identical can only be done easily if the parts are derived from a single source combined with formatting or other options to create the output.
These pre-compiled output files can themselves become checked-in source that is only rarely built when the core definitions change.  Why precompile EVERY time right?
File Dependency Modules
The real nightmare of make-files is keeping dependencies straight.  When done manually, they create a kind of straight-jacket that eventually ties you in knots.  I envision tools that inspect the source tree for how files are related.  It could be as simple as looking for #Include statements but it could get quite sophisticated as well.  Imagine just building each function in a separate file with no references to any headers or other modules of your code.  The dependency tool simply figures out based on the symbolic analysis of the modules what files it depends on.  Of course the modules would be language specific but they could also be invoked further down the build process like at link-time.  The output of these tools would be dependency description files.  Date/Time stamps can be used to optimize only re scanning things that changed and naturally, these intermediate files can be checked-in as well.
File location based on dependencies
I have long thought of this as one of my cooler ideas.  Suppose you had a set of directories titled Layer0, Layer1, ... where Layer0 constitutes all the base OS APIs you have at your disposal - things that you know will always be there and you can't change.  Above that in Layer1 is only code that depends on Layer0 code.  Linking this code need know nothing above Layer1 to work.  Layer2 is just code that depends on Layer1 and lower code, etc.
If  you make a change that adds a dependency to an equal or higher layer, the file gets moved to the higher layer automatically, or a warning gets issued and it won't build till you put it in the right place.
Programmers become very aware of how their changes effect the dependencies of a project and they instinctively know that the lower order layers are likely simpler and faster bits of code.
I would think such a system will have a natural ordering to it that will promote faster and better factored code.
Compilers to a common AST structure
An AST stands for Abstract Syntax Tree and is an internal structure that compilers use as an intermediate form of the code as it parses the source code.  What if you could output the AST in text form?  You could see how the parser is interpreting your code and errors would be much better understood if you could see the associated AST.  What is nice is that this intermediate form of the build process could also be used to help editors do smarter editing and syntax errors would have so much information to present on a problem as to make it very clear, most likely, what is wrong without needing to create really great error messages for the problems.
What is even more powerful is the ability to write parsers for multiple languages that output the exact same AST format.  Now we have cross-language support without runtime or target machine dependencies!
And, as suggested in other areas of this article, the AST files could also be checked in and the re-parsing of unchanged files skipped in the build process.
AST to target object compilers
The next phase involves taking input from the AST files and building linkable objects or IL (Intermediate Language) output files.  It's not a long move from AST form to IL or Linkable form - it could be a very fast process.  Optimization options go in this phase of processing I should think.
Run-Time and Link-Time tools
I recall trying to better understand .obj files sometimes to help me understand link dependencies better.  We just didn't have the tools.  But it seems at this point one could start testing, profiling and debugging your code before its even compiled into final form.  Unreachable code could be found and dependencies analyzed here.  These tools could feed back into the file location based dependencies by noting where and how many link ends there are.  Graphical models could be constructed to see dependencies across modules and suggestions made on better ways to pull parts together into modules.
Final Compilation and post compilation optimization
Another pass can be used to construct stand alone .exes or .dlls or IL assemblies.  Here is where massive configuration options can be used for proper packaging.
I have always hated COM because of the way it forces you to create so many layers of overhead that you might never really need.  COM has a great way of abstracting things out so you treat everything the same but many times its just not needed.  I have often wish I had the choice of how to connect with other code - use COM with IDispatch interfaces, use COM with core interfaces, use direct DLL export linking, or just paste the code into my code.  These are all legitimate ways to go, I wish I had a world where I could just use all of them together and change them as needed.
Build analysis tools
How long did each phase of the build of parts of a big project take?  How many times did the same module have to be linked in for others to use?  Can I see a dependency map of all parts of the process from start to finish?
Graphical Programming
Finally, the ultimate dream - building software like legos.  If there were a nice way to represent code via a hierarchy of ORM models with different shaped knobs on them to represent interfaces.  You pick up a piece and you instantly see a list of other parts it can connect to and you just graphically drag them together to form a new part.
It's all a dream but some ideas that I hope might inspire someone to think of a way to do this better.
I'm not a compiler writer so I can't really claim any real expertise here, but it just seems we could do things better.