Monday, November 10, 2014

Play Framework Essentials, Learning Scala, and Functional Programming Principles in Scala

I've been really busy over the last six months or so:

First, I was a technical editor for Play Framework Essentials. It's a fun book because every example is first shown in Scala and then in Java, which makes it kind of a Rosetta Stone for the two languages.

Then, I was a technical editor for Learning Scala. That was a great book! Jason did a great job making a really complex language very approachable. This wasn't the first book I've read on Scala, but it's certainly one of my favorites!

Finally, I took Martin Odersky's class on Coursera, Functional Programming Principles in Scala. I really liked that as well because it gave me a much stronger grasp of functional programming fundamentals. Furthermore, I participated in a study group at Twitter, which made the whole experience much more rewarding.

My hope is that I'll get to the point where I can teach a few Scala classes at work. Hence, I've been working my butt off two hours a day on BART and a couple hours on Saturdays in order to get up to speed with Scala. I still haven't written anything large, but at least I'm feeling a lot more comfortable with it these days.


Wednesday, June 11, 2014

Python: Custom App Labels in Django

Django has had a long-standing missing feature that made it impossible to give your apps friendly names. See #3591 and #14251. Thankfully, this will be fixed in Django 1.7, but switching to the latest version of Django is not an option for me right now. This was making my admin look really ugly because I have app names such as "mps" and "budget_cost_data". Those would show up in random places in the admin as "Mps", "Budget_cost_data", "Budget cost data", etc. What I wanted was "MPS" and "Budget Cost Data".

There are many ways to try to solve this problem. You may be able to switch to Django Suit or Grappelli and then use localization to fix it. However, when I tried that approach with Django Suit, the breadcrumbs were still broken.

I finally found this snippet which got me started down a viable path. I grabbed a copy of the Django egg I was using, unzipped it, and copied django/contrib/admin/templates/admin to my project's templates directory. Then I started hacking. I created two filters:

@register.filter(name='custom_app_label')
@stringfilter
def custom_app_label(value):
  """This is used to change the Django app labels.

  See: https://snipt.net/chrisdpratt/overriding-app_label-in-django-admin/

  """
  return settings.CUSTOM_APP_LABELS.get(value.lower(), value)

@register.filter(name='custom_title')
@stringfilter
def custom_title(value):
  """This is used to change the <h1> in admin/base.html."""
  if value == "Site administration":
    return "Site Admin"
  if value.endswith(" administration"):
    try:
      app, administration = value.split()
      app = custom_app_label(app.lower())
      return "%s Admin" % app
    except ValueError:
      pass
  return value

Then, I applied one of those two filters everywhere in the admin templates that I needed to. Make sure you remember to load your filter library at the top of each template you modify.

You may want to give your models friendly names as well, such as:

class PlatformBudgetCostData(models.Model):
  ...
  class Meta(object):
    verbose_name = "Platform Budget Cost Data"
    verbose_name_plural = verbose_name

At the end of the day, I got something that wasn't as beautiful as, say, Django Suit, but at least the app names, titles, capitalization, etc. looked right.

Friday, June 06, 2014

A Blog Post on Excel, 4x4s, Moonshots, and General Purpose Software

Using Excel is like driving a 4x4. Writing custom software is like building an interstate freeway.

With a modest amount of training, you can do amazing things with a 4x4. You can go to places where there are no roads. If you don't like where you are, you can just as easily drive somewhere new. The downside is that when you're driving off-road, you generally have to drive pretty slowly, and you have to bring your own gas.

You do have to be a specialist to build an interstate freeway. In fact, it requires an amazing amount of work from a large number of people. Yet, once it's built, a 100,000 people a day can fly by at 70mph. It offers them tremendous convenience; they don't even need to bring their own gas! On the other hand, a freeway is only useful if it goes in the direction you need to go.

Hence, Excel offers tremendous flexibility, whereas custom software offers tremendous convenience. Excel is easy to get started with, but grows unwieldy as you scale the size of your models and teams. Custom software is fairly difficult to get started with but is easier to scale in terms of size and usage.

I have to interact with Excel a lot, and yet I've spent most of my career writing custom software. What amazes me about Excel (or, rather, spreadsheets in general) is that it's so amazingly general purpose. Think of how useful spreadsheets are to businesses all over the world. There are few pieces of software that are as general purpose or as widely useful as spreadsheets are.

To some degree, it seems to me that Silicon Valley is drunk with the power of being able to write custom software. Pretty much my whole career has been spent writing it. But what about writing new general purpose software like Excel? As far as I can tell, not many people get to work on general purpose software, and even fewer people get to create new types of general purpose software.

The first commercial spreadsheet was VisiCalc. What an amazing invention! Although VisiCalc was used by far fewer people than, say, Excel, it's notable because it represented a new "type" of general purpose software; it changed the world.

Now, think about all the other types of general purpose software. Most of it was invented a long time ago. Word processors, operating systems, web servers, databases, ERP systems, etc. were all first created decades ago. That's ancient history as far as computing goes! (Although, to be fair, all of these things are incredibly recent in terms of the history of civilization or the history of mankind.)

So, where are the new types of general purpose software? It's actually kind of hard to recognize them because they haven't been around long enough or copied enough times to be recognized as a "type" of software. However, I think that we're far enough along to recognize Facebook is an instance of the type "social network." Similarly, Twitter is a "microblog" (among other things). Both of these things have had a fairly worldwide impact.

Facebook and Twitter also illustrate another thing. You could say that both of them are really just custom software, whereas the truly general pieces of software that they're built on are web browsers, web servers, databases, programming languages, and operating systems. On the other hand, as I said above, they're also instances of new types of software: social networks and microblogs. Innovation is like that. An instance of one type of thing can become its own new type of thing. Innovation is always pushing toward greater specialization.

Now consider programming languages. We've seen a lot of evolutionary improvement, but a lot of the revolutionary ideas first happened decades ago. Think of subroutines, OOP, exceptions, processes, threads, coroutines, types, recursion, closures, compilers, interpreters, virtual machines, etc. These concepts have been around for decades. Sure, we have new programming languages that each have a different philosophy on how these things should work, but most of the really big ideas happened decades ago. I thought software transactional memory might be a counterexample since it seems fairly recent, but even that started about two decades ago.

Imagine, if you were to invent a fundamentally new replacement for subroutines, create a new way to package data and/or code together, or create a new form of concurrency, what would it look like?

Of course the flip side is that it's really hard to get people to understand and accept fundamentally new ideas. It took at least a decade or two before OOP was generally considered a useful technique, and that's still not universally accepted ;) The same is happening now for functional programming. And who knows where we'll go with concurrency in the future!

Truly new ideas just don't come along that often. It takes a while for a new instance of a thing to be recognized as a new type of thing. There aren't that many "moon shots" as Google likes to put it. And when they do come along, they often take a really long time before they gain widespread acceptance. Furthermore, the first mover often isn't the one who reaps the greatest rewards. Excel wasn't the first spreadsheet. Google wasn't the first search engine. Facebook wasn't the first social network.

I'm not really saying anything new. Kevin Kelly did a much better job explaining all of this in his wonderful book, What Technology Wants, but I'll leave you with a quote from another excellent book, The Myths of Innovation, Don't worry about people stealing your ideas. If they're any good, you're going to have to cram them down people's throats!

Saturday, May 31, 2014

Security: Scam Involving the "assoc" Command on Windows

My dad sent me the following:

Today I received a call from a Mark Atkison. He claims to be with Windows Technical Services, located in (or on) Brainbridge Island, Washington. Phone number 206-201-2413

Mark claims for the last two weeks my computer has been downloading online infections, junk files and miscellaneous viruses. I asked him about my “online ID number” Mark said my “customer license Security Identification number is: 888DCA60-FC0A-11CF-8F0F-[deleted]“. Mark said I could verify this by pressing the Windows key and r at the same time.... That would open a “run box” When the run box opens I was to type ASSOC. When I hit the Windows key + r, I saw a box open with “cmd”... which I figured stands for “command”. If I remember correctly, I erased the “cmd”. I was to type ASSOC. When I did, I saw something come up with “exe”. By the way, when I typed in ASSOC, I would not hit enter. I thought this might be some kink of scam or bull shit. I told Mark I was going to contact my son who is a high end programmer. Mark said I could call him back at the number listed above and refer to, “Docket number Yash 120695”. Mark told me they will show me the error and warning reports they have been receiving from my computer or lap top operating system.

This evening, I looked up Brainbridge Island, Washington... I found there was no Braindridge Island, Washington. There was however a Bainbridge Island, Washington (no “r”). Did I make a mistake? I'm not so sure I did. I had him spell out everything. I did a Google search for the phone number he gave me.... I found the following:

Match Found! We found phone number (206)201-2413

See Full Results

Received a call from (206)201-2413? View the comments below or add a comment of your own for 2062012413. Remember to not reveal personal information. Tell us about 206-201-2413. What time did they call and what was the call about?

Anonymous Monday, 19 May, 2014 15:19
Yes this is a scam call, beware do NOT install anything on your computer. They will records all your personal info

Anonymous Monday, 19 May, 2014 15:12
was this a scam call???

Anonymous Friday, 16 May, 2014 16:00
They told me windows was receiving a virus report on from my computer.

I think my instincts were good and your assessments were right on. Needless to say, I will not be calling Mark.

Best wishes to you and yours,
Dad

Apparently, the assoc command in Windows can be used to change file associations. The attacker could use this to convince you to treat .txt files as .exe files. Then, he could give you an executable that has a .txt extension. You would think it was safe, but when you opened it, it would run the executable, thereby taking over your computer.

At least, that's what I think is going on. I'm not 100% sure. It kind of seems like a like of work for the attacker since it involves him calling people manually.

Friday, May 30, 2014

Being Turing Complete Ain't All That and a Bag of Chips

I was talking to someone the other day. He said that given two Turing Complete programming languages, A and B, if you can write a program in A, you can write a similar program in B. Is that true? I suspect not.

I never took a class on computability theory, but I suspect it only works for a limited subset of programs--ones that only require the features provided by a Turing machine. Let me provide a counterexample. Let's suppose that language A has networking APIs and language B doesn't. Nor does language B have any way to access networking APIs. It's entirely possible for language B to be Turing Complete without actually providing such APIs. In such a case, you can write a program in language A that you can't write in language B.

Of course, I could be completely wrong because I don't even understand the definitions fully. Like I said, I've never studied computability theory.

Friday, May 16, 2014

Raspberry Pi: Building an LED Digital Clock

As I mentioned in a previous post, I really enjoyed reading Programming Raspberry Pi: Getting Started with Python. One of the chapters in the book teaches you how to build an LED digital clock. It took some futzing around, but I finally got it done :)

The first problem I had was that I didn't know how to solder. My buddy Chris Dudte gave me a kit to learn. I watched a bunch of YouTube videos with the kids in my Raspberry Pi class, and then we put the circuit board together. Problem solved.

The next two problems I encountered were with the author's library for talking to the smbus for controlling the LEDs, i2c7segment. One of the problems resulted in my saying quite a few less than charitable words under my breath. The Python code kept giving me the error message "IOError: [Errno 5] Input/output error".

I finally figured it out. On line 42 of i2c7segment.py, the code is hardcoded to use smbus.SMBus(0). However, sometimes you need to use smbus.SMBus(1). You can run "sudo i2cdetect -y 0" and "sudo i2cdetect -y 1" to figure out which bus to use. You should see "70" in one of these two. The hardest part of figuring out this problem was that I thought the software must be correct, and that I must have wired it or soldered it wrong. My guess is that this might be a Raspberry Pi model A vs. model B thing.

To hack around the problem, I edited i2c7segment.py to use bus number 1. A better approach would be to allow the calling code to pass a bus number or to try to autoselect it; however, I didn't bother.

Anyway, I'm super excited that I got the project done, especially considering this is the first time I've ever done something like this. Special thanks to my buddy Chris Dudte for all his support and for giving me a Raspberry Pi in the first place!

Wednesday, May 14, 2014

Best Practices for Software Engineers

As I mentioned in my last blog post, I'm going to be giving my "Best Practices for Software Engineers" talk at both the East Bay Ruby Meetup and at BayPIGgies (the Bay Area Python Interest Group). We're planning on broadcasting the BayPIGgies meeting using a Google+ Hangout on Air. If you're interested, here's the event, and here's the direct YouTube link.

Thanks to @nicholsonjf for setting this up!