HelloWorld

Welcome to the world of static blogs! I've been neckdeep in a seemingly stable and wonderful project called nikola, that allows for some pretty fantastic python static site generation.

I've been using flask for all my "deploying a quick webapp to openshift" needs, which has worked out splendidly, but after experimenting with http://thisweekinfedora.org I felt like I had to dive in head first and see what all the hubub was about.

Some 4-ish hours later, here we are!

Some caveats:

  • I'm git push -f openshift master to a fresh php5.4 cartridge
  • The amazing themes listed at bootswatch.com are unvailable to me...
  • I don't even know what I don't know I'm doing wrong yet ;)

BUT!

I'm SUPER excited to play around more with programmatically generating static content with the power of python.

Shout-out lmacken, threebean, and ryansb for their wizardry and patience.

What can we do to improve computer education?

SIGCSE 2015 for Computer Science educators kicks off this year from March 4 - 7 in Kansas City, Missouri.

Headshot Pamela Fox

The SIGCSE Technical Symposium addresses problems common among educators working to develop, implement and/or evaluate computing programs, curricula, and courses. The symposium provides a forum for sharing new ideas for syllabi, laboratories, and other elements of teaching and pedagogy, at all levels of instruction.

Last year Pamela Fox, Computing Curriculum Engineer at Khan Academy, was part of a panel on called "Disruptive Innovation in CS Education." I spoke with her afterwards to get her thoughts on how open source fits into education and the future of computer education.

This is a partial transcript.


opensource.com

Where are you from?

I was born in Los Angeles, grew up in upstate New York. My dad is a computer science professor at Syracuse University. My mom is a rocket science programmer. My dad is launching a "big data" MOOC, so we're both very interested in this field.

Where are you now?

Now, I work for Khan Academy in Mountainview California and live in San Francisco. I went back to the west coast as soon as I could and joined Google after graduation from University of Southern California in Los Angeles. I went to Australia, then got back to bay area three years ago. I was working on Google Maps API in Developer Relations, writing articles and demos, which is basically what I do now, but for non-proprietary technology.

I first learned HTML in the 7th grade. Within a year, I made a website that taught HTML to other people called "htmlforkids" or something (even though I was a kid too.) That was probably my first "official" educational content. After that I was a computer camp counseler. In college, I organized workshops around 3D programming (started a SIGGRAPH chapter). I use Khan Academy to get better at math now.

Why free and open source software?

I really enjoy teaching, and I enjoy trying to figure out how to teach something. I find it fascinating when I put out new course, I read the comments and say, "Wow, I forgot what it was like not to know." I'm interested in humans, I read a lot about how humans work, and behavioral science. There is a lot of that in teaching people, is all about learning. I'm just learning.

I'm generally a fan of open source, and that is another reason why I'm at Khan Academy, where we do that. As a web developer, I shouldn't have to reinvent the wheel. Often I say, "Really... really? I gotta solve this problem? I'm the only one that has ever tried to do this?" No, it's just someone didn't share it. Many of these components should be open source. Some people may say, "Well, we don't have jobs if we don't have to rewrite it." I don't want to believe that we live that way. I have friends with open source projects, who have tried to make money on it, doing enterprise versions, and getting paid for support. I'm always interested in the different ways of monetizing code. I feel like that part is still has open questions.

I think we should encourage sharing, kids are used to the idea of "cheating." Someone copies your code, and they say, "Hey, that's cheating" and we have to tell them, "No, it's MIT Licensed, and it is open source."

We have to teach them that sharing is OK. We have to do a better job of teaching open source and sharing, counter to what they may see in school. I'm upset there is no representation from the coding academies here at SIGCSE. They are trying to figure out how to get people ready for programming jobs in 12 weeks. I feel like I'm here representing that industry. Half engineer, half educator. I feel like I'm representing the "meritocracy" getting a real job thing too.

What can we do to improve computer education?

Coding academies are formed by people who learned alternatively, or didn't do well in college, and they are figuring out how to teach based on industry, and their good and bad experiences in college. They have good things to say about career oriented computer science education. They should be here (at SIGCSE) too. Girl Develop It, Women Who Code, they are all doing similar work, and they are disconnected from this world. I'm not just trying to do women-friendly hackathons but newbie-friendly hackathons too. More women are newbies than men right now, so if you fix things for newbies—people who are intimidated, who don't think of themselves as superstars—you fix it not only for women, but for men that have that same situation. Right now, we have to say "this is for women/girls" but the lines are getting increasingly blurred, and maybe we won't have to worry someday, but for now, we have to bring stuff up to parity good stuff.

I'm quite interested in how we can prepare the next generation for the world with it's concerns about security and privacy. I like reading books like Little Brother by Cory Doctorow, which is a YA book that forces kids to think about these issues. I want to find a way to introduce the next generation to these issues and be relevant. If anyone has ideas on how to do that, I'd like to know.

Lead Image: 
Share
Rating: 
(8 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

bitHound puts out features, not fires

The following is a partial transcript from a phone interview with Dan Silivestru, CEO and co-founder of bitHound.io—automated, open source, code quality analysis software.

Where are you from originally?

I was born in Romania, lived there for six years, don't remember any of it. Then my parents went to Israel for seven years, then moved to Montreal, Quebec, and lived there for another seven years. Then I moved to Ontario, and I'm still here.

When did you start bitHound?

We started in November 2013, and I went full-time in January 2014.

How many folks are at bitHound now?

There are nine of us. A CTO, COO, development team of four, plus staff to handle operations and HR.

What is bitHound, and what do you do?

We are centered around the concept that writing code is easy, but building resilient, remarkable software is difficult. There is much that can be told as you go along. We analyze projects from conception to today, pointing out hotspots that require attention, and suggestions on how to fix them. We track code as you move forward, so we can say if things are getting better or worse.

Something I'm proud of is a feature that showcases the dependencies that your projects have from npm and Bower, for example. It helps you understand the code that you bring into your project, and then rank it from a quality perspective. The dashboard shows you up-to-date or out-of-date status, as well as assigns you a bitHound score that is derived from Code Quality, Maintainability, and Stability. You can then pick better dependencies based on quality level. You can really dive in with bitHound.

Does bitHound support other programming languages besides JavaScript?

It is JavaScript only for now, but in the future there could be more. Rather than just the bare minimum, we think that to provide value, we've got to do a deep dive into a language. We run almost a dozen different "critics" or analyzing engines, to get "actionable insight."

(Remy: Completely understandable. Source code analysis is not what I would call a "trivial" problem...")

It is not an easy problem, it takes a lot of time and effort. We've been at it for for about a year, and it is still in closed beta.

What is it like for a bitHound user?

We strive to make the user interaction with product very simple. We think that if your software needs a manual, you're probably doing some things wrong.

The experience is simple: use OAuth for GitHub. You enable bitHound on a per-repository basis. We run our analysis, and it takes 2-20 seconds, and then we fill-in the timeline going backwards.

The idea was, on the first dashboard, you would get an "eagle-eye" view: the top five priority files, and you can expand the list further. We've had many users who are new to the concept of quality concerns such as linters, duplicate functionality, etc. So, rather than presenting an overwhelming amount of information, we present the top five most worrisome files and annotate the code with issues, so you can filter and address them. You can see on your dashboard, which dependencies are out-of-date, and we have some upcoming security analysis features in the works too.

(Remy: This sounds like it would be useful for researchers. Students in my HFOSS course at RIT have to do repository analysis as part of our "Commarch" assignment each semester.)

We have some students who use our products, and are getting introductions to professors. It seems only recently that source control is even being taught at the college level. When dealing with JavaScript, which doesn't really benefit from compiling, linters are a life-saver. The students really appreciate it. We consider these very simple things.

A big part of what we want to do behind bitHound is answer: "How can we get people to build quality code?" You have to treat your job as a craft. It is craftsmanship, and proper tooling around making software.

When did software craftsmanship become a passion for you?

It is one of those things where you get burned. You get burned in production once, twice, and then again. Then you say: "How did I get here?"

I'm self-taught from a software development process. Much of what I learned was learned "on the fly." Having gone through institutions, some left me better, some worse. The first five years of my career was focused on delivering features on time. Then I got introduced to this concept of: "If you are going to cut corners, you need to document it." When we started doing tests, though the upfront work was higher, six months later, we saw big benefits. Even as systems got more complex, you have safe-guards in place. You can go back and fix it. We were able to keep our bug-count down.

We were putting out features, rather than putting out fires.

Per feature costs, we're much lower. In the long-run, it allows your organization to move forward at a steadier, and faster, pace. Then again, at other places I've joined, they were on their second or third full rewrite. It didn't happen overnight. I didn't just wake up and say: "Test test test." It wasn't until after getting burned...

How did you get into software development?

At the University of Waterloo, honors science, then honors physics. Then I took time off to make money. While I was working at a company, I had a friend working in IT, while I was working on phones. I asked him: "IT? What do you do?" He showed me AS/400 systems and greenscreens. I asked: "How do I do that?" And the next thing I knew, I was sitting in front of the VP asking to do it.

I got a AS/400 manual, and the opportunity and big break to do that. I did that on my own time for a few weeks, and after a few months there, I said: "This is the career I want" and never looked back. I had some tremendous mentors along the way. I was there for a few years, then went into "e-business" doing consulting.

What makes a good mentor? Where and how do you find them?

The number one trait for mentors I've had, to this day, was selflessness. They were doing it for the pure joy of helping someone else develop their craft. They are not about: "I'm teaching to get something out of you for free." Obviously, they have to be knowledgeable, but you can tell more about them as a mentor by how they carry themselves.

If the first five years of my career was about: "How do I code?" After that it was about defining components that interact together. Later on, after my first consulting position, I had a new mentor, with new questions.

Dan: "We should write tests... Why?"

Mentor: "How do you know what the interface between components really looks like?"

Mentors have to be good at their craft, but you can tell a lot by the questions they ask. Listen to how they go about development and the way they ask questions.

What does your day-to-day look like?

I wrote quite a bit of code when we started, but I'm sure much of it has been rewritten. Since we announced funding in late November, it has been more about investor follow-up for me. I've matured within the code environment and in the running-a-company environment. Mostly I'm steering the ship. Day-to-day, lots of emails, some interviews like this one, working with team to set priority/strategy, and yes, I still write some code. I'm not anywhere near the critical path anymore though :P

What are your feelings on free and open source software / FOSS?

I was a co-founder of a company, tinyHippos, that we founded in 2009—which was acquired by Blackberry in 2011. One of the visions, was open sourcing the Ripple Emulator. That happened, and it was fantastic. There were only three of us that moved over, in terms of "how you run a project in FOSS."

I'm proud that they took this project, and donated it to the Apache Foundation. It sits side-by-side with PhoneGap. There was great experience in "how to foster a community" and "how do you be a BDFL?" (benevolent dictator for life). Someone who moves a project forward in a community.

We've loved FOSS throughout our career, and use it constantly at bitHound. Our analysis depends on many popular frameworks. JSHint for linting, esprima, async for callback structure, ZeroMQ for distributed parallel computing platform. You can check out our talk at JSConf last year about distributed complex computing.

Any others?

Yes! We make use of over 80 open source projects throughout our solution but a few that come to mind are d3, jquery and Polymer in production.

Where is bitHound contributing back?

Right before starting this company, Gord Tanner was the core contributor to the Apache Cordova Project; he created of the Ripple Emulator, which he donated to the Apache Foundation and is used today by Microsoft, Intel, Adobe, and over 250K developers. He unified the platform in a coherent manner, and still contributes there. For bitHound, he is a co-founder and CTO leading the technical development of services.

bitHound has simple philosophy, while we're in heavy product build, is about product, but we have come across projects in our stack that have issues. We always contribute any fixes or additions back into that project. That is our standard operating procedure. If we need to make a specific change for use that has no benefit to community at large, we don't push it, but if we fix a bug or feature, we contribute it back upstream always. That is a recommendation to any company out there. If you are going to get something for free, from someone's hard work, then if you enhance it, you should contribute it back so all can benefit. Otherwise, the community would die if everyone consumed and no one contributed.

Internally, we have components that we think will be beneficial. There will be a prominent links on our site to our GitHub.

One component is what we call "The Farm" where we have workers assigned to do work in parallel. A simple event bus really, with aggregate results coming back. We're all often dealing with the single-threaded nature of the JavaScript language, and you'll see us trying to open source that.

One thing that has happened internally with The Farm, is we've already abstracted into a separate project, to be a released. This is one of the things I've realized in my career—all of us have—just putting something on GitHub and calling it "open source" is not enough for it to take off. It must be prepared and ready for the community. That means proper README, proper docs, proper instructions for looking at project and contributing, beyond downloading and installing. Anyone can npm install but it is a different thing to make it so that contributors can understand how to augment. We will be taking our time putting our code out there, because we wanna do it right.

Any final message or parting thoughts?

Don't just write code, consider what you are doing as a craft. It takes time, and practice, and it takes time to build something that is resilient and beautiful. Open source is a great way to perfect that craft. One reason we built the dependency tool into our product was to get more people diving into code they can contribute to. Seeing other people's architectures, will expose you to better approaches.

This is sort of what spawned bitHound.

Software development is a craft, and you should be proud of it. Take the time, learn the craft, and strive to build masterpieces—the ones that gather great attention in the community. We're in this to do more than just make money, and bitHound will be free forever for open source, with no restricted features. We're huge believers in this movement. We participate, and we want to help.


This work by Remy DeCausemaker is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. Special thanks to @itssamlowe for her contributions and edits.

Lead Image: 
Rating: 
(7 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

The elements to a better future for software

In this interview, I take a deep dive into the life and motivations of Kyle Simpson, an open web evangelist and the author of the book on javascript, You Don't Know JS. Find him on GitHub and see his many projects and posts on Getify.me.

Where are you from?

Oklahoma City, born and raised. Started school in Oklahoma, but now based in Austin, Texas—since mid-way through college. I live there with my wife and two kids. I moved to Austin because there wasn't much of a tech community in Oklahoma back in the 90s, and Austin was the nearest big tech hub. Now, I go back to Oklahoma to visit and see they have a fantastic community there, and I'm jealous! It's great to see!

Where did you go to school?

I started at University of Oklahoma, then transferred to and graduated from Texas State University with a B.S. in the engineering track of Computer Science.

What is your day-to-day like?

I have two different kinds of days; days where I speak/teach, and days where I do FOSS development. On teaching days, I'm connecting with community, and teaching JS to make a living—mostly in a corporate workshop environment, or public workshops associated with conferences. Those days I stand all day, and teach, and lecture, and walk people through exercises.

When not on the road doing that, I'm participating in the FOSS community—writing code, or writing blogs, or books. I spend lots of time doing that, constantly on GitHub with commits and pull-requests flying everywhere. I currently have a 300+ day streak going on GitHub—not to show off—but to inspire others to do more, and more regularly, with FOSS contributions. If my streak can encourage one person do one extra contribution, that's what it's all about.

The best way to describe it: 50% of my time I spend teaching to pay bills, and 50% donating time to the FOSS community, to build awareness around the web platform and its technology with the theory of "all boats rise with the tide." The more people who learn and appreciate web tech, the more people will hire me to teach it to them. I'm an avid learner of things, and the best way to learn is to teach others. I think "how can I make this make sense to others?" As soon as I learn something, I write code to explain it, find a book or post to describe it in, and if I find something I didn't understand, branch off, and learn more, then start the cycle again. It just gets deeper and deeper and deeper.

I said a while back that, "I think it is important for developers, especially those breaking into industry, find ONE thing you love to learn, and master it." It may not be the one thing you write all the rest of your code with, but it is the process of sticking with something to mastery that is valuable. Don't just jump from thing to thing to thing. While you may get a good paycheck doing that, there is something missing from the art of deeply understanding something. Once you've accomplished that, and you know what there is to know, then branching out to try things is great! Be looking while you branch for that next thing you want to master, rinse, and repeat. Constant jumping around as a "jack-of-all-trades-master-of-none" was more relevant 5-10 years ago. What is missing now is people who really know what they are doing.

Our industry currently rewards "flexibility" and working at the whim of someone else. "Yesterday, we wrote everything in Angular, and today, we're going to rewrite everything in React..." After enough of those inflections, you "become" a senior developer, but you miss out on appreciating a technology in the way it really deserves with deep understanding.

Mastery? How?

Well, specific answers are variable. Angular will be much different than Node. In general, the important skill is the curiosity and desire to learn. Don't just read a line of code and say, "I guess that is just how it works..." Keep reading, and keep following the rabbit hole down until you can say you understand every part of that line of code. I tell my workshop attendees that I don't expect you'll write your own framework, but that you could. Don't treat frameworks as black-boxes—you need to understand them intimately. If you choose something, know how it works but also WHY. Knowing when to change comes from understanding why—not because there is a great book, or how many "stars" the repo has. Those are poor signals. Beyond understanding of the open source community, your own understanding is the strongest signal.

You don't have to reinvent the wheel, but you should understand how the wheel rolls before you decide to bolt it onto the car you're building.

How did you get started in FOSS?

I was working for a company, not as a developer, but as a "User Experience (UX) Architect." I worked in project management team prototyping User Interfaces (UIs), and handing them off to the dev team. Inevitably, everything I wrote was just put into production, or adapted slightly. I was working on a project in 2008 that needed to make cross domain Ajax requests, and back then it was a real pain. I needed a solution to prove out my concept for the app, and I said, "I know some Flash, and I know that it can do that." So I built a JS API wrapper around an invisible flash file, with the same API as the XMLHttpRequest (Ajax) object, and I called the project flXHR (flash based XHR).

Once I got it working, I thought, "Maybe other people will find it useful?" so, I released my code as open source. Back then, open source was pre-GitHub, so source was all on my website, and I pointed people at it from blog posts, etc. I also put code on Google Code too, but there wasn't as much of a community back then either. In early 2009, I wanted to get into conference scene. 2009 was the first big JavaScript-specific conference, JSConf, and so I decided to go and speak about SWFObject (one of the most downloaded projects on the web at the time), which I was using heavily in flXHR. I was a core dev for SWFObject and gave a "B track" talk at the conference. Only like three people showed up to my first talk, but I fell in love with the idea that I could speak to call attention to open source code and inspire others to help make it better!

The fullness of my open source perspective came later that year, in November of 2009. I released the project I'm probably most known for: LABjs (a performance-optimized dynamic script loader). I gave a talk at JSConfEU in Berlin Germany about script loading. Two hours before going on stage, I was overhearing lots of people talking about this new site called GitHub, so I went and signed up while I was sitting in the audience. I pushed all my LABjs code there, and that was my first official: "I am in the FOSS community" moment.

One thing you wish undergrads would be exposed to before the leave school?

Unquestionably, "Simple Made Easy," a conference talk by Rich Hickey, who works at Cognitect on the Datomic database as well as the Clojure language. He's a completely brilliant dude. The talk is so important to me, I don't have just have it in a bookmark, but on my toolbar and reference it practically daily. The premise is, there are two terms that people conflate: "simple" and "easy." He actually compares "complex" versus "hard." The root word for complex comes from "complected," as in strands of rope being braided together. Highly braided code is complex, and harder to maintain and refactor. Software developers, when building, they focus on building "easy" software—that is easy to install and use, and does a lot for you. That pursuit often results in complex software.

If developers go after modular simple (non-complex), non-braided software, they can often end up with easy software too. If you go after easy, you usually end up with complex, but if you go after simple you can also achieve easy.

Node.js is my example. I was trying to install it on a Virtual Machine, and had the operating system (OS) requirements, but couldn't install it because the OS didn't have the proper version of Python. Node, a JavaScript framework, uses Python for its installer? Why do I need Python to install Node??? The answer was because writing a cross-platform installer in Python was easier... but when you add additional braiding, you can also make it more complex to implement and maintain.

Nearly every framework on the planet claims to be modular, but most are not. Modular, to me, means that a piece could be removed, and the framework would still be able to be used. "Separate files" does not a modular framework make, if all those pieces are required for the framework to work! My goal, my desire, is that developers go after simple modular design, and that be the most important ethic. What comes from that then, is proper design that can be made easy for people to use. We need to stop worrying as much about creating pretty-looking, "easy" interfaces, and instead worry a lot more about making simple software.

What is your toolchain?

Sublime is my text editor. In principle, I love browser-based editing, but I run the nightly versions of my browsers, to find bugs early while I still have a chance to get them fixed. I can't handle browser crashing and uncertainty for when I'm writing code.

Sublime has so many plugins you can use for whatever you want. Though I don't use many, other people like the "intellisense" plugins, and many other plugins that are part of a great ecosystem they've built.

My other main tools are the browser developers tools in Firefox and Chrome.

My other mission critical tool is the Git command-line tool. GitHub is my graphical git client, because it effectively augments my usage of the git CLI.

Git. How do you use it?

I don't have a lot of fancy process, it depends on whether I'm writing a book or writing code. For books, when I make a change, I want to write a coherent section, and make one commit per section. In the writing of one section, I may add to the Table of Contents, or clarify another section, or add another. Whenever I have a logical series of changes, I git add each individual file, (files written in markdown BTW), and git commit -m. In the commit message, I list which book the commit portrays to, which chapter(s), and a quick description of what it was about. The commit history of the book series really tells a story in and of itself, of how over months I figured out how to write them, section by section, reorganization by reorganization!

I typically use git commit -m ".." && git push, so that I push right after committing.

It is not often I do batch committing, usually only when I've been on an airplane without wifi for awhile, in which case I'll push 5-8 commits at a time once I get back online. Usually, I try to push right after I finish the section I'm working on.

For code, I have two different strategies. If it is a "big" feature, I create a feature branch, and I put several commits into the feature branch. The goal isn't to finish the feature and do a massive merge, but to regularly merge. I like to develop in stable batches, merge regularly, and don't do harm to master. If I do make a bugfix on master while developing a feature, I rebase the feature branch to get that fix in. I don't necessarily do short lived branches, but I do short lived differences. :)

Many devs do squash merges, and want to appear to have "Dreamed up this perfect feature and written it perfectly all at once." I don't want that. I want to preserve the history. In rare cases with a pull-request that has lots of individual commits that are all logically connected, I'll do a squash-merge.

In cases where I have a simple bug fix to make, I'll generally just add and commit directly to master. Regardless, Every time I'm doing the final commit, I'm committing both the docs and tests. I Firmly believe that it isn't DONE until it has docs and tests. I don't really do Test Driven Development (TDD), but Test-oriented or Test-informed development. I have a set of tests, and sometimes they are written ahead, but the typical plan is "I don't know how it should behave" when I fix something with a new feature--it will take me working through implementation to know. I develop the tests along with the code--code and test--rather than writing code after tests or the other way around.

I'm much more formal when working on other peoples' projects, or as a bigger team. I try to stay away from scenarios where I need the complicated cherry-picking or interactive rebasing features of Git. I've done those things only a few times in my career. I use GitHub for most of those things, and it handles those cases pretty well. A pull-request with 2-3 commits, whatever their process was, is something that is useful to preserve in the history, so I'll usually just merge it as-is.

What are you currently working on?

Other than my books, I have three main areas of project interest I cycle through on any given week.

Number 1 that gets most of my interest is asynchronous ("async") programming patterns (promises and generators, that sort of thing). I have a library called asynquence, a promises-style asynchronous library. It can also handle generators, reactive sequences, and even CSP. (see: Hoare's seminal book "Communication Sequential Processes") with these higher-level patterns layered on top of the basic "sequence" abstraction. Most other libraries have just one flavor of async programming, but I've built one that can handle all the major patterns. I think async is one of the most important things that JS devs need to get up to speed on. I've got several conference talks and projects about that topic.

We're recognizing more and more that sophisticated programs need more well-planned and capable async functionality. Callbacks alone don't really cut it anymore.

[Remy Decausemaker: Yes, I reckon this jives well with Python incorporating Tulip and features from Twisted into the core librarystarting with Python 3.3.]

Number 2 is in the same vein as the "compile to" languages for JS. Experimentation is important for the language. Taking that to it's extreme, I have a set of tools to define custom JS syntax and transpile to standard JS—basically, standard JS + custom syntax. I'm working on tools that do "little" transformations on your code. The bigger picture is "inversible transforms" or able to transformed in both directions, non-lossy transformations. If you can define them two-way, you can have your "view" for your own editor, and a "view" for the team repository. You check code in and out, and you can work on code the way your brain works, and the team can work in the way their's does.

When you use CoffeScript for example, it is a lossy transformation, and an "all or nothing" decision. Everyone needs to work on it in this way, or not at all. The simple version of what my tools can do is simple stylistic things like spaces versus tabs. The tools can change that code style for you instead of just complaining with errors.

ESRE is one such tool I'm building for two-way code-style transformation.

let-er is another tool that transpiles a non-standard version of JS block-scoping into standard JS block-scoping. I have a series of in-progress prototypes of these various tools, and eventually I can go back and write the overall "meta" tool that drives them with the two-way transformations.

Number 3 is a crossover between JS/CSS. It is a project in the templating world. There are two extremes in templating; zero-logic templating or full programming language templating. Zero-logic templating includes projects like Mustache. We don't want business logic in the views, so we use no logic at all. But in practice, this creates very brittle controller code that's closely tied to the structure of the UI, and that brittle connection is precisely what we wanted to avoid by keeping the concerns separate.

The other extreme, is you have a full programming language in your templating. My metaphor is "if I hand you a pile of rope, I can teach you to build a rope-bridge, helpful, or a noose, which isn't quite so helpful." If you are in a "15-minute-must-do-feature crunch" you'll just drop in if-statements and function calls, and then put a TODO comment to fix it, but then you rarely do. That's how we unintentionally leak business logic into our views.

Neither extreme is good enough. We need something in the middle, that has enough logic for structural UI construction, but keeps out all the mechanisms that you can abuse to do business logic.

For 4-5 years, I've experimented with a templating engine that is a happy medium, called grips. It has enough structural logic, but is restrained so that you can't do things like function calls, math, etc. It's mature enough that I use it in my projects and have rolled-out production websites with it. It is definitely a work-in-progress, but it is "stable enough" to be used. People still to bikeshed about the syntax for sure and may not like the choices I made. But I think I at least asked the right questions, like: What does a templating engine need or not need? I started with nothing and only added features when it was necessary to do structural stuff. You have basic looping and conditionals, but in limited fashion. I summarize that balance as, "if you find yourself unable to do something, it should be a signal that you don't need it in your templating engine."

Two years ago, I started watching the rise of LESS, SASS, and other tools like COMPASS. What struck me was how limited they were in solving the problems I thought were important in CSS world. Those tools require the CSS to be recompiled every time you make a change. "Compile a HTML template, re-render with external data" is a solved problem. For some bizarre reason, it didn't occur with CSS though.

So, I invented grips-css, a CSS templating syntax similar to LESS, on top of the core grips templating engine. Most importantly with grips-css: data is external (i.e. CSS variables), which means all the data operations that projects like SASS are inventing declaritive syntax inside of CSS to handle, instead you can and should do those data operations outside of CSS, producing new data and then just re-rendering the template.

If I wanted to change "blue" to "red," I don't need to recompile all my CSS, I can take my pre-compiled CSS, and just re-render it with the different variable data.

The compiled CSS template is basic JS, which means you have the option of re-rendering CSS dynamically in the browser on the fly, for example responding to changing conditions with CSS. It's much cleaner to simply re-render a snippet of CSS and inject it into the page than to use brittle JS code to change CSS style properties. Of course, you can also run grips-css on the server much like you currently do with current preprocesors. The point is you have both options with grips-css, instead of being limited to server-only and inefficient total recompilation. What I'm trying to do is suggest that the spirit of what SASS and the others are going for is good, but the way they are going about it is limited and not terribly healthy for the future of CSS.

CSS templating is, I think, a much cleaner and more robust way to push CSS tooling capability forward.

You mentioned important problems to solve in CSS? What are they as far as you are concerned?

Three main things were solved in LESS. Variable data, that can be changed and reused, structural things like mixins to achieve DRY coding. And, extends, which is a light version of polymorphism to override pieces of templates. We needed to solve those things, and they did, but as I said, we solved this in text templating years ago, and we should apply those same principles from HTML/text templating to the world of CSS. There's no reason CSS needs to invent its own solutions for these problems.

So, what is next?

Putting on the "prognosticator hat," what do I think we'll see in the next 3-5 years?

Applications are going to become "UI Optional." The new Apple Watch has a pretty limited display, and some apps won't show anything at all. Things like Google Glass, or Oculus, you'll have apps that don't have any visual representation at all. This is what I call the coming "APIs-as-Apps" era. Your "app" might be nothing more than a piece of code that can send and receive data—a distributed API. We have some companies that build apps that care greatly about branding. Twitter wanted you to experience their app the way they wanted. Facebook wanted you to experience the Facebook app the way they wanted. But there is a reality that people will experience apps without your UI at all. Companies must give up control of the presentation, as our devices and interactions with them diversify from purely visual to audible or tactile interactions.

My watch may read things to me without UI, and that is nothing more than a data operation. Facebook should provide the text for my watch to read to me. The UI doesn't necessarily go away, but it becomes an optional add-on to apps. In the longer term, I'd like to stress the decoupling more. We see people building single-page, complex, front-end driven apps. Most of the app is in the front-end. Gmail is cool to use, sure, but I don't think they are very flexible in that new optional-UI trend. It will be hard to separate Gmail the App from Gmail the UI.

Developers are making assumptions about access to unlimited, fast bandwidth with every retina image served up... We're not designing things in layers the way we know we should be. For all the people on slow connections it's just, "Meh, they'll get better access eventually." We need to give users tools in the browser to choose what is important to them. I should be able to say, "No, I don't want a huge single page Gmail app, I need a simple post-in-page mobile version." This is much more than just expecting a "mobile site." We need layered sites.

We need to take a serious look at how much we assume that UIs and data bandwidth usage are an unlimited resource. This could be like "responsive 2.0"—responsive not just to screen layout, but to network conditions too. The app should figure out that I am roaming and not shove everything at me it possibly can. UI needs to be decoupled, simplified, layered, and more focused on portable apps.

I heard a conference talk years ago from PPK (Peter Paul-Koch). He suggested, "Why is it I can't send a text to share an app with you? Why do you have to buy it from an app store?" He proposed that monetization would shift from the app to the data. He believes apps should be self-contained portable pieces of code that can be freely shared around regardless of device. JS is great for this because it is ubiquitous. For instance, if Facebook wanted to charge me for data, because there was no UI on my device within which to serve ads to me, I should be able to decide if I want to pay them for the data of my updates.

I hope that kind of thing represents the future of the web and the usage and consumption of apps.

Lead Image: 
Open source code for a better food system, code with grass image
Rating: 
(9 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

Open source all over the world

After a full day at the annual meeting of the Opensource.com Community Moderators, it was time for the the last item on the agenda which simply said "Special Guest: TBD." Jason Hibbets, project lead and community manager for Opensource.com, stood up and began explaining, "In case it wasn't going to happen, I didn't want to say who it was. Months ago I asked for any dates he'd be in town. I got two, and picked one. This was one day out of three weeks that Jim was in town."

The moderators, in town from all over the world for the All Things Open conference, stirred at the table. Their chairs squeaked and snuck a few inches edgewise.

"We're going to get a half hour to hear from him and take a couple questions," said Jason.

The door opened, and as if it had been waiting for him the whole time, the only vacant seat at the head of the table was soon occupied by a tall fellow.

"How is everyone doing?" said the man. No suit, just a button down shirt and slacks.

The next tallest man in the room, Jeff Mackanic, senior director of Global Awareness at Red Hat, explained that the majority of the Community Moderator team was present today. He asked everyone to quickly introduce themselves.

"Jen Wike Huger. Content Manager for Opensource.com. Happy to have everyone here."

"Nicole. Vice president of education at ByWater Solutions. We do FOSS for libraries. I travel and teach people how to use software."

"Robin. I've been participating in the Moderator program since 2013. I do lots of stuff for OSDC and work in the City of the Hague, maintaining their website."

"Marcus Hanwell. Originally from England, I'm now at Kitware. I'm the technology lead on FOSS science software. I work with national labs and use things like Titan Z doing GPU programming. I've worked with Gentoo and KDE. Most of all, I'm passionate about joining FOSS and open science."

"Phil Shapiro. I administrate 28 Linux work stations at a small library in D.C. I consider these folks my coworkers and colleagues. And it's wonderful to know that we can all feed into the energy and share ideas. My main interests are how FOSS intersects with dignity, and enhancing dignity."

"Joshua Holm. I spend most of my time staring at system updates and helping people search for jobs on the Internet."

"Mel Chernoff: I work here at Red Hat, primarily on the government channel with Jason Hibbets and Mark Bohannon."

"Scott Nesbitt: I write for many things, but have been using FOSS for long time. I'm a 'mere mortal' just trying to be more productive, not a sysadmin or programmer. I help people meld FOSS into their business and personal lives."

"Luis Ibanez: I just joined Google, but I'm interested in DIY and FOSS."

"Remy DeCausemaker: Resident Hackademic at the RIT MAGIC Center and Adjunct Professor for the Department of Interactive Games and Media. Been writing for Opensource.com for about four years now."

"You teach courses for the new FOSS Minor then," said Jim. "Very cool."

"Jason Baker. I'm a Red Hat cloud expert, mostly doing work around OpenStack."

"Mark Bohannan. I'm with Red Hat Global Public Policy, and I work out of Washington. Like Mel, I spend a good deal of time writing for, or finding folks from, the legal and government channels. I've found an excellent outlet to discuss positive things happening in government."

"Jason Hibbets. I organize the organized chaos here."

The room has a good chuckle.

"I organize this chaos too, you could say," says the brownish-red haired fellow with a gleaming white smile. The laughs grow then quieten. Breaths become baited.

I sat to his left and had a moment to look up from transcribing to glance up. I noticed the hint of a smile behind the knowing eyes of a man who has led the company since January 2008, Jim Whitehurst, president and CEO of Red Hat.

"I have one of the greatest jobs on Earth," began Whitehurst, as he leaned back, crossed his legs, and put his arms behind his head. "I get to lead Red Hat, travel around the world and see what goes on. In my seven years here, the amazing thing about FOSS, and, broadly open innovation, is that it has left the fringe. And now, I would argue, IT is in the same place that FOSS was in its early days. We are seeing FOSS going from an alternative to driving innovation. Our customers are seeing it, too. They're using FOSS not because it is cheaper, but because it provides them with control and innovative solutions. It's a global phenomenon, too. For instance, I was just in India, and discovered that, for them, there were two reasons for embracing of open source: one, access to innovation, and two, the market is somewhat different and wanting full control.”

"The Bombay Stock Exchange wants to own all the source and control it. That is not something you would have heard five years ago in a stock exchange, anywhere. Back then, the early knock on FOSS was that it was creating free copies of things that already existed.' If you look today, virtually everything in big data is happening in FOSS. Almost any new framework, language, and methodology, including mobile (though excluding devices), are all happening first in open source.”

"This is because users have reached size and scale. It's not just Red Hat—it's Google, Amazon, Facebook, and others, they want to solve their own problems, and do it the open source way. And forget licensing—open source is much more than that. We've built a vehicle, and a set of norms. Things like Hadoop, Cassandra, and other tools. Fact is, open source drives innovation. For example, Hadoop was in production before any vendor realized there was a problem of that scale that needed to be solved. They actually have the wherewithal to solve their own problems, and the social tech and principles to do that. "Open source is now the default technology for many categories. This is especially true as the world moves more and more to content importance, such as 3D printing and other physical products that take information content and apply it.”

"We have this cool thing in one area, source code, but it is limited. But there are still many opportunities in different industries. We must ask ourselves, 'What can open source do for education, government, and legal? What are the parallels? And what can other areas learn with us?'"

"There's also the matter of content. Content is now free, and we can invest in more free content, sure. But we need free content that has a business model built around it. That is something that more people should care about. If you believe open innovation is better, then we need more models."

"Education worries me with its fixation on 'content' rather than 'communities.' For example, everywhere I go, I hear university presidents say, 'Wait, education is going to be free?!' The fact that FOSS is free for downstream is great, but the upstream is really powerful. Distributing free courses is great, but we need communities to iterate and make it better. That is something that a lot of different people are doing, and Opensource.com is a place to share what is going on in this space. The question is not so much 'How do we take content?' as it is 'How do you build and distribute it? How do you make sure it is a living thing that gets better, and can morph for different areas?'"

"But the potential to change the world is limitless, and it's amazing how much progress we've already made. Six years ago we were obsessed about defining a mission statement. We started by saying, 'We are the leader,' but that was the wrong word, because it implied control. Active participant didn't quite get it either... Máirín Duffy came up with the word catalyst. And so, we became Red Hat, the company that creates environments to agitate action and catalyze direction.”

"Opensource.com is a catalyst in other areas, and that is what Opensource.com is about. I hope you see yourselves this way, too. The quality of content then, when we started, versus now, is incredible. You can see it getting better every quarter. Thank you for investing your time. Thank you for being catalysts. This is a chance for us all to make the world a better place. And I'd love to hear from you."

I stole a glimpse of everyone at the table: more than a few people had tears in their eyes.

Then, Whitehurst revisits the open education topic of conversation again. "Taking it to an extreme, let's say you have a course about the book Ulysses. Here, you can explore how to crowdsource a model and get people to work together within the course. Well, it's the same with a piece of code: people work together, and the code itself gets better over time."

At this point, I get to have my say. Words like fundamental and possibly irreconcilable came up when discussing the differences between FOSS and academic communities.

Remy: "Retraction is career death." Releasing data or code with your paper could be devastating if you make a mistake. School has always been about avoiding failure and divining 'right answers'. Copying is cheating. Wheels are recreated from scratch ritualistically. In FOSS, you work to fail fastest, but in academia, you invite invalidation."

Nicole: "There are a lot of egos in academia. You need a release manager."

Marcus: "To collaborate, you have to show the bits you don't understand, and that happens behind closed doors. The reward model is all about what you can take credit for. We need to change the reward model. Publish as much as you can. We release eventually, but we want to release early."

Luis: "Make teamwork and sharing a priority. And Red Hat can say that to them more."

Jim: "Is there an active role that companies can play in that?"

Phil Shapiro: "I'm interested in tipping points in FOSS. It drives me nuts that the Fed hasn't switched to LibreOffice. We're not spending tax dollars on software, and certainly shouldn't be spending on word processing or Microsoft Office."

Jim: "We have advocated for that. A lot. Can we do more? That's a valid question. Primarily, we've made progress in the places we have products. We have a solid franchise in government. We are larger per IT spend there than the private sector. Banks and telcos are further along than the government. We've done better in Europe, and I think they have less lobbying dollars at work there, than here. This next generation of computing is almost like a 'do-over'. We are making great progress elsewhere, but it is concerning."

Suddenly, the door to the room opened. Jim turned and nodded towards his executive assistant standing in the doorway; it was time for his next meeting. He uncrossed his legs, leaned forward, and stood. He thanked everyone again for their work and dedication, smiled, and was out the door... leaving us all a bit more inspired.

Lead Image: 
Rating: 
(7 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

You don't know Javascript, but you should

This is a partial transcript of a meeting with Kyle Simpson, an Open Web Evangelist from Austin, TX, who's passionate about all things JavaScript. He's an author, workshop trainer, tech speaker, and OSS contributor/leader.

Thank you all for having me. I'm Kyle Simpson, known as "getify" online on Twitter, GitHub, and all the other places that matter. I was here in Rochester teaching a workshop for the Thought @ Work conference this past weekend, and figured I'd stick around to check out some JavaScript (JS) and Node classes here in the New Media Interactive Development program, so thank you for having me.

I have been writing a book series on JavaScript called You Don't Know JS. The entire series is being written in the open, up online on GitHub for free reading. They're also being professionally edited and published through O'Reilly. There are five titles planned for the series: two have already been published, the third is complete and in final editing, the fourth is almost complete, and the fifth one will commence soon.

  1. Scope & Closures: Covers closure primarily, which is one of the most important foundational topics. All JS programs use closures, but most developers don't know that they're using it, or what to call it, or just how it works.
  2. this & Object Prototypes: Covers the mystery of how the this keyword works, and then tackles the misconception that JS has classes—not true! Instead, JavaScript has prototype delegation, and we should embrace that rather than trying to fake class orientation.
  3. Types & Grammar: Goes deep into coercion, the mechanism most people think is evil in JS. I encourage you to dig into it and learn it, because coercion not only isn't as bad or weird as you've been told, but it can actually help improve your code if you learn how to use it properly!
  4. Async & Performance (in progress): Explains why callbacks for async programming are insufficient, then goes deep into promises and generators as much better async patterns. Also covers optimizing and benchmarking JS performance.
  5. ES6 & Beyond (planned): Covering all the changes to JS coming in ES6, as well as forward looking to beyond-ES6 evolution on the horizon.

To understand the spirit of this series, compare it to JavaScript: The Good Parts by Douglas Crockford. His book was both good and bad for our community. It's almost single-handedly responsible for bringing lots of developers to (or back to!) the language and giving it serious attention. We owe a lot to him for that. But it also taught developers that there is only a small part of the language you need to learn. And because you only have to learn a little bit of it, that's all most developers ever learn. Even developers with 5 or 10 years JS experience know comparatively very little of the language.

My books are the opposite. They're the anti-"The Good Parts." That doesn't mean they're the bad parts, it means they're all the parts. Rather than avoiding most of the language because one guy said to—rather than running away from the hard parts—I encourage you to run towards "the tough parts" and learn them. When you see something in JS that you don't understand or is confusing, instead of blaming the language as being poorly designed, turn your attention toward your own lack of understanding, and spend the effort to increase your understanding.

This is somewhat unique to JS developers, that they expect a language should be so simple and intuitive that merely glancing at it should be enough to understand it, and that if they can't, it's a failure of the language. Expecting perfectly self-explanatory syntax and rules wouldn't be reasonable of any other language, like Java or C++. If you were confused by code, you wouldn't blame the designers of those languages. You'd blame either your own understanding, or at least that of the person who wrote the code. Either way, learning the language better is the best solution to that lack of understanding. Many times, when developers hate something about JS, it turns out it's because they simply don't understand it enough. When I explain how it works, many times they go from hating it to appreciating it—and by the way, appreciating doesn't mean liking, it just means respecting.

I believe JavaScript takes time to learn properly and completely, and that if you're going to write it, then you should invest that effort. You should understand why the code that you write works the way that it works. Instead of saying "it works, but I don't care how," the most important question you can always ask is: "How does it work, and WHY?" I'm an open web evangelist who teaches JavaScript for a living. I work with developers all the time who've learned JS incompletely and improperly, and they're having to fight hard against the grain to re-learn it. That's why I'm so encouraged to see you learning JS in university. By learning JS properly here in school, you can graduate and come right into the industry as a new generation of developers that already understand and appreciate the importance of JS as the standard for the entire web platform.

JS is going to be the foundation of the web platform for the rest of our careers. We might as well get to know it better!

I'll leave you with this: I believe strongly, that the most important thing you can learn at university—of course, you're being taught lots of great stuff—but the most important is how to learn, and how to love and enjoy learning. You'll never find "just one thing" you love and do that for the rest of your career. The industry reinvents itself every couple of years. If nothing else, it'll just be Apple doing that. You have to be adept at learning and remastering new things. That's the path to success in your career, whatever interests you dig into.

Q&A session

Q: Five books, should they be read in a specific order?

A: Scope and Closures is in most demand and chronological release order is certainly OK. The first three are about the core of JavaScript. Four and five will be build upon the first three, but mostly deal with new things coming to the langauge as of ES6.

Q: How important is free and open source software in your work?

A: Everything about my career is open source. I believe very strongly in the power of open source, and its position in the future success of our industry. If you study the history of technologies, they start closed/proprietary, are shepherded through adoption and evolution, and eventually end up open. Ultimately, open always wins. But increasingly, I believe open should be the default mode. Many people say, "I don't feel like I wanna put my stuff out, they'll make fun of my crappy code..." And when I write code, people say "you just have more confidence 'cause you're good." But if you look at my old code, there is some terrible stuff in there. When I say "you" in "You Don't Know JS", that's a collective term. I don't know it either.

Every time I start writing code for a project, I start with an empty file, publicly, on GitHub. I do the best I can and am constantly evolving. But instead of just using GitHub as a platform for marketing my own code and ideas, I assume that every line of code I write is the worst, and the only way to get better is with the help of others. Open source collectively makes the best software better than any one person can make.

It is a culture you should strive for individually, and professionally. I believe very strongly, "open" is the reason why all this exists, and why the stuff we're doing now will still exist 10 years from now.

Q: I'm in that camp where I am afraid to code publicly. Where do I start?

A: My perspective—and there are different answers—is to seek out others' projects. There is a lot of FOSS contribution that isn't about code. Docs are usually left to the end of a project, and are neglected, but it is critically important they are up to date. If you can read others' code, and add details, examples, or tests, that is a super important contribution you can make. Many of "the rockstars" in FOSS got there by just pitching in and started with docs/tests. Some projects go the extra mile, and identify "low hanging fruit" or bugs that are known to have simple solutions. It is a great place to start with, and you can learn about how the project works. Even providing bug reports is a way you can contribute without writing your first line of code. But even one line of code is important. Someone after you can learn from it.

Q: Where?

A: GitHub is the de facto standard. Any community is fine, sure, and I wouldn't say "pick this project." You should pick a project that is interesting to YOU. If you are into  data visualizations, get into D3. Find what you are passionate about. If you do, you'll quickly build your confidence, and that will create a virtuous cycle of making both the people and code, better.

Q: You said that you think JS will be the "only language for the web" for our careers? I'm not necessarily a supporter of Dart, or other similar languages, but do you not expect those to succeed?

A: Great question, and loaded, but... Dart isn't going to succeed in replacing JavaScript, not because it is bad or poorly designed, but because of how Google is going about it. Going beyond what they say on their site, they've positioned it to compete against JS, in hopes of replacing it, rather than being a language that experiments with things to intentionally inform and influence the future of JS. From the original "leaked memo" where the world learned about Dart in terms of "fundamental flaws [in JavaScript] that cannot be fixed," to the Dartium VM they're building in Chrome to sit alongside JS, to the Dart2JS transpiler—the messaging is unclear and smells of not just being a "better compiling JS lang," but more an attempt to hope JS declines if developers can just write Dart in the web natively. I can tell you this for sure: Mozilla will never implement Dart in Firefox. Unless there is a future where Firefox doesn't exist, which I cannot imagine, Dart will not replace JS.

In a bigger sense, there are hundreds of languages that you can compile into JS. You want to run your code on the web, so can "transpile" it into JS. I don't like most of those languages personally, but they are all super important! Source code, is not for a computer! There are an infinite number of ways to write code to produce the 1s and 0s. Source code is for the developer, and you need to find the language that works the best with your brain. Also, we need more experimentation, and more Compile-to-JS languages, like CoffeeScript, which influenced many great things being added to JS in ES6. The future, I think may be limited for CoffeeScript itself, but that's OK because it was very important to evolve JS forward. As far as Typescript, I don't like classes, but Eich is on record saying there may be something like the type annontations in the future of JS.

Learn JS first, but as you go about your career, you'll find other languages that work better for certain problems or teams. Many people do that because they don't wanna learn JS, but that is the wrong way of going about it. Once you really know JS, then it's totally OK and healthy for you to find other languages that you prefer that will use JS as their compliation target. That's great for the future of the web platform.

Lead Image: 
Javascript code close-up with neon graphic overlay
Rating: 
(10 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

If you write code, this is your golden age

This is a partial transcription of the two keynotes from day 1 at the All Things Open conference in Raleigh, NC held on October 22nd and 23rd.

Keynote from Jeffrey Hammond, VP & Principal Analyst at Forrester Research

If you are a developer, I cannot think of a better time in the history of our industry to be a developer. It is a golden age if you write code. If you have people work for you that write code, there is another part of that message. You need to understand how open source is part of that process or you risk being consumed by generational changes happening in our industry.

When I built my first company in 1999, it cost $2.5M in infrastructure just to get started and another $2.5M in team costs to code, launch, manage, market, and sell our software. So, it's not surprising that typical "A rounds" of venture capital were $5-10M.

If you look at where we are today, ideas cost about 90% less than what they cost when I got started in this space. Today, omni-channel clients, deployed on elastic infrastructure, aggregate discrete services, use managed APIs, integrate open source software, employ devops techniques, and focus on measurable feedback.

Open source is so ubiquitous. Anyone want to go talk to a purchasing officer when they want to spin up another node? That is driving the adoption systemically. We're seeing the evolution of DevOps, and open source is a driving force for modern application development.

We ran a developer survey at Forrester; 700 developers across multiple countries. We asked: "What classes of open source software have you used in the last 12 months?"

  • 41% open source databases
  • 38% operating systems
  • 34% web servers

When we look at developers that built cloud, mobile, or big data, the response rates are at near-unanimous levels:

  • 93% cloud
  • 92% mobile
  • 78% big data

1 in 5 developers have not used open source software. Even the folks using Microsoft and Oracle are using open source software in some way.

In the past, we've seen servers being the big one. Now it is the open source databases. We're starting to see a shift in the DB domain. We've seen a tail off in app servers. As we see more modern development, they don't use app servers but change how they run a backend.

When we look at modern applications, we see differences from previous generation. We see APIs everywhere. Developers today look for services and APIs first. There are more services now to use and call asynchronous communications. For the last 12 years, we've been locked into a MVC (model, view, controller) world. Tightly coupled MVC architectures. That doesn't work as well now. You see more event-driven frameworks now. Many of the old frameworks are as well suited for today, and the new ones, are open source software.

Lightweight process communication frameworks such as Socket.io, Nginx, and Node.js are replacing previous open source software and tech. It isn't just open source versus proprietary, we're seeing open source versus open source, and substitution happening. (i.e. Subversion versus Git). In memory, databases are becoming more and more popular and have broad adoption. Elastic infrastructure is the norm. No more "max" licenses.

We are seeing an emerging dominance of sharded SQL or NoSQL databases for open source options in that space.

Behind modern apps, we see "modern engagement architecture." Systems that are built on top of systems, going to employees or customers. Web apps are building in a very different way; fourth tier architectures. In between, there is an aggregation tier, that gathers and ingests data in real-time, from the IoT (Internet of Things), and other sources, and predicts what is the next best step via context. After that, a delivery tier, with things like Amazon Web Services.

Engagement platforms such as Netflix, can be decomposed, you can see all the pieces—client tier, delivery tier, aggregation tier, and services tier—all with open source sprinkled throughout. The same is true of Evernote, Instagram, and Untappd... you can see the pervasive use of open source in modern application architectures.

Increasingly, that innovation is driven by open source software communities. Collaborative collectives set to drive innovation in the industry forward. Those of us looking to hire developers have to understand how to restructure to work with and build on top of the work of those collectives, to attract talent.

I've been asking the question: "Do you write code outside of your day job?" And, 70-75% of developers say "Yes." Some only a couple of hours. Some as much as 11-20+ hours a week off the clock coding. That desire to write code on your own time is driven by a number of motives—learning, starting a company—but those motivations are intrinsic, it makes the developer feel good and feel happy. 1 in 4 developers tell us they contribute to open source software as part of their own time. Those developers are some of the most talented and creative developers out there. If you think of development as a creative field, then you know it is widely distributed. If you are looking to hire talented developers, productive developers, there is a correlation with those that do FOSS (Free and open source software), and those you want in your organization.

There is such a lack of top-talent in our space, everyone is looking for folks who understand modern frameworks and NoSQL, and can build on top of the cloud. If you have those skills, and know how to use the frameworks to build apps, there is a very bright future ahead for you. In the US, we expect developer growth of over 28% until 2020. In 2014, they had a $92K average salary, and in 2013 it was $70K. This shows there is a demand/supply imbalance.

We are in a generational tech shift. Modern tech is different than client/server applications. We gotta understand how to use this tech, and elastic architectures that allow us to innovate cheaply. The cheapness of open source is a perfect fit for modern platforms. 4 out of 5 use open source, and it works.

Open source software projects drive the collaborative collectives, whether they hang on GitHub, or in Drupal, or in foundations like Eclipse or Apache. These are the centers of gravity of development moving forward into the next decade, and the center of gravity grows.

Talent is a seller's market, and we are in a golden age.


Keynote from Dwight Merriman, Executive Director and Co-founder of MongoDB

Hi everyone. I'm Dwight. I work at MongoDB, as mentioned, and I've been there from the start. I wrote a lot of code in the version 1.0 days. I wanted to talk this morning about something consistent with what Jeff was talking about.

Pre-object oriented programming. Pre-relational databases. It goes back a long way. I'm going to focus on the data layer. I do like that term "modern application" because we're not building the same things. It's not "we need a new version of inventory system" but an entirely new class of apps, B2B, and B2C that didn't exist before. The way we build them has completely changed. The schedule this morning, you see all the talks, and they are peppered throughout with the names of projects and products, and lots of open source things. One thing I was thinking about was how Jeff talked about elasticity of licenses. In my mind, it is a big deal for "granularity" of software. It is easy today to mash up third party code that is from completely separate products. You can imagine using a dozen third party components or more. It can become a very long list.

If it was closed source, it would be hard to have that many different things to go buy, evaluate, and develop. That granularity aspect is very real and a big part of how we write software today. We don't want one big monolithic system, we want to break up what we can. Different specialized pieces is what we want, and that isn't as easy if it's not open source.

In that context of modern apps, I wanna talk about data. "Big data" and "NoSQL"—these are weird and imprecise words. There is something big happening. We're in the midst of the biggest change in the data layer of IT (information technology) in 25 years. There is something big going on. Big data is a bunch of new technologies in the data layer that are happening right now; and unusually large amounts change. A boring but accurate way to describe this is, you can divide out the buckets. NoSQL, scalable and great for building modern apps. Hadoop, more on the analytics side.

All this is happening, and it is a big change. The types of apps we write is changing too. In regards to that, the new apps and usecases we work on, the shape of the data is different. The shape of the data. The data is unstructured, polymorphic. It isn't just tabular. Accounting data in the real world is tabular. 1-to-1 mapping. Real-world usecase data, is all varieties of shapes. Unstructured on one side, and complex, or plastic and evolving on the other. The new tools are great at dealing with this. JSON, and the document-oriented notion of the database, or message passing in that format. This is significant in my mind. Saying "unstructured" data is inaccurate. Usually it has structure, but it's dynamic. It is like the age of dynamic/static-typed programming languages. The data we are dealing with is dynamic, and I like to think of it that way.

I like to think about the size too. Massive scale. How to deal with that. Computers I have today are cheap, and not very tall. They scale horizontally. In 1999, we spent $100M on hardware... we were serving lots of ads, but the computers were 1000 times slower, and they cost more money back then. You could get a "big" computer back then, but they were just taller. Today, you can't get that much faster processor by buying a "bigger" computer. You gotta scale horizontally. Parallelism. That dovetails perfectly with cloud computing, and the imperative to scale out, and this is really the only way to do it.

The main thing I think about is latency. The notion of the application and use of nightly or daily reports in your mailbox... that is old-school at this point. People are used to real-time interactions in the services they use on their phones. As developers, as we build systems, we gotta start from a real-time mentality. I would tell my team, of the 100 apps we build, default to real-time. As a CIO, that may sound obvious, but that is a big deal! Think back to Computer Science education in the 1980s. You are supposed to default to "batch" in 1980, batch algorithm complexity was lower, and the way to go. Much faster than real-time. Today, things are so much faster, and can handle it.

If my delivery mechanism was the mail, or FedEx, then batch processing wouldn't matter. One of the properties of modern applications, and something we gotta do. Anytime I do a new app, I default to real-time, and not only when there is a reason not to. Many of the new tools, NoSQL, facilitate that too.

Shape, size, speed, approach

Approach is "how we write code," and that has changed a lot. It is no longer architect, code, design, test, deploy. There is just constant iteration. On the developer side, it is helpful. And for the business side, it is great to be able to say: "Is this what you want?" and it can be changed rapidly to converge on needs if it isn't. It has always been hard to spec things. This opens doors to creativity. Even if I could write a perfect spec, I can have ideas as soon as a month in! You get a better outcome. That is a big part of apps today.

Look at Facebook. Everyday the site changes. Daily. For years. Iteration. That is a great real-world example.

Applications

I'll give some specific examples. The usage of MongoDB, when I'm asked is "general purpose database" or "operational database." We are trying to take a new cut at what is an operational database in the context and methodology of today. IoT is going on, and this is one place where people use NoSQL, but people are doing lots of things. John Deere is doing some neat things with data these days.

One of John Deere's messages is: "Feeding the planet"

Imagine it is harvest time, and we're running the combine, and as we go around, you can measure yield. You can measure how many pounds of stuff went into the bin in the last 1-5 seconds. You know where you are by GPS. We can have a yield map, topologically, and very precise. Of course, that is a lot of data. What can we do? We can look at it, and visual representations of it, like heatmaps. You can see problems like "the river is over here." You can imagine a very granular—something the size of this podium—being low yield, and using point-by-point fertilizer! That is the kind of stuff they are working on, and these are real-world tangible things. Candy crush is "cool" but it is intangible. This isn't just in the mental domain.

One of Bosch's messages is: "Connecting The Planet"

They have a business unit that does lots of software for automation and industrial management. They have drills or riveters—a power tool like the one in your garage—except these cost $15K and put rivets in airplanes. Safety is critical in airplanes. There must be quality control. Are they going in right? We can look at it, rivet by rivet? You can see if there are three mediocre rivets in a foot, which is no good, but over 100 feet, that could be ok. The gun records all that data, and it becomes part of the quality control of the manufacturer. In these cases, you gotta store a lot of data. That is what makes this interesting, the shape of the data. The timestamps that have nuggets of info. The polymorphism. All the sensors, not all of them the same, and how well you can deal with those differences.

One of Edeva's messages is: "Safeguarding The Planet"

Think about self-driving cars, which generate tons of data. You can use that data to do analysis to improve traffic flow, and figure out where to change something, or add a road, and what is the problem. You can use it for safety analysis. They've built a technology when you are driving across a bridge in Sweden, they see your speed. If you are going the speed limit, it does nothing. If you are going too fast, the system creates speedbumps just for you! It creates depressions in the road! If there is an accident on the bridge, you can ruin traffic for half-a-million people in a day. It has been a very successful project for them. I have mixed feelings about not being able to speed... but that is good for them.

(audience laughter)

Looking at these examples, you can get broader than that, but the question I would put out is: "What are you working on?" Maybe not this second, but over the next year. Are there intersections, or creative things? We've seen great things come from startups, like Uber. But I've also give you some examples here from more "classical" organizations. There is more to do in the "old" domains and fields, and we have the tools now to adopt the new mentality. Be ambitious, try to tackle these things.

Lead Image: 
Rating: 
(10 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

Head of Open Source at Facebook opens up

What is seen hereafter is a partial transcription of James Pearce's OSCON session Rebooting Open Source at Facebook.

For hundreds of years, open has trumped closed—sharing has trumped secrecy.

In a humble way, this informs our program at Facebook. We have 200 active projects at Facebook, with 10 million lines of code. Many hundreds of engineers working on these, with over 100,000 followers and 20,000 forks. We contribute to a wide range of projects (i.e. The kernel, mercurial, D, etc). We've even open sourced the designs of our data centers and machines in the open computer project. We want to share a collection of things we've learned along the way.

Why is this so important?

The reason, open source is dorm-room friendly. Our roots stretch to a young undergrad in 2004 who picked the FOSS (free and open source) software that was available, the classic lamp stack. Our capacity to participate in communities to make a better place has increased.

When we find a piece of open source software (OSS), we first try to scale that, and then find the limitations of a project. So we try to improve them and make them work in scaled environments, and we see this pattern happening over and over again. Mark's decision to use PHP, for instance, had limitations. We built the HipHop "compiler" HHVM project, and even more recently, the PHP enhanced language called Hack, launched back in March. Data, web, infra, front-end, all of our technology stack. It is closely aligned with our hacker culture, and how our organization was perceived. We asked our employees...

"Were you aware of the open source software program at Facebook?"

  • 2/3 said "Yes"
  • 1/2 said that the program positively contributed to their decision to work for us

These are not marginal numbers, and I hope, a trend that continues.

A large number of those people said their experience using our projects in the open helped them get ramped up prior to being hired. That is a huge win for our company.

This is important part of why open source is valuable to our company. And you need to be able to articulate the value.

#0: Always articulate the value FOSS brings to your company

There are always costs and investments, so understand what your return is. Naive ideology only goes so far, you need data to support continuation. We're confident it helps us do a better job. It helps us keep our tech fresh, justify architectural decisions, bring more eyes to our code. Open source is like the breeze from an open window; it keeps things from going stale.

But, if you wind the clock back a year, you'd find this three20 project, which has been discontinued... Our PHP SDK... deprecated. Our fork of Memecache, with a description of "test" and commit messages of "5" "6" and "7"...

*audience laughter*

This is the "throw it over the wall" syndrome. We're guilty of this, I'm sad to say, and it is almost worse than not doing it at all.

You need to continue to care about the things you release, or how can you expect others to care about them?

#1: Use your own open source

It is essential to continue using the version you release. Don't create internal forks, keep the code fresh, keep working on it. The community will notice if you don't. Eat your own dogfood.

Sometimes you'll have to integrate your open source code with closed/proprietary tools internally. It usually means you create plugins, or adapters, and make architecture decisions that make your project better. Presto, we needed it to integrate with both open and internal databases. We had a strong plugin architecture, and plugins for open databases, and then plugins for our internals.

Nevertheless, we weren't doing that well last year. We decided to refresh our team and get our house in order. At that time, our web team open sourced React at JSConf. React is one of the most exciting projects in the Javascript world in the past years, with a great community response. It reminded us at Facebook that we knew how to get great projects out there. That initiative came from the developers themselves. There was no promotional team internally; they came directly from engineers.

#2: Decentralize project ownership

Make sure the engineers are the sole custodians. External engineers work with internal engineers directly. No monolithic structure. As we looked at the reboot, we needed to figure out what we already had, and getting the portfolio under control.

We needed to answer 3 key questions:

  1. Which projects did we own?
  2. Who contributes?
  3. How healthy are they?

Most were on Github. Github of course had a great API, so we wrote a script (in hack) to access and enumerate over projects, and get:

  • every repository
  • every commit
  • every pull request
  • every issue

So we stored all this data, and put it into MySQL.

I love Github, but I find it easier to use SQL to filter what is going on. We found some things to address. We realized we could do this import process again and again, and see how trends evolve over time. I am now one of the world's experts in the Github API throttling mechanism, and we've got it running very efficiently. All of this is to implement two things: instrumentation and publishing.

#3: Invest in instrumentation

We now have time series data and can create metrics. This is Argus and shows the total number of watchers over time. Up to 100,000 followers, and polling every minute, we can watch over time, and we can find inflection points which GitHub didn't have. We launched an iOS library called Shimmer, and then tweaked it, and those surges can be seen after investing in the iOS community. Being able to monitor and publish data and progress, it shows that we are being disciplined, and can get respect via empirical data.

We have over 35 metrics we follow.

Five most important metrics:

  • Average number of Followers
  • Number of Forks per repository
  • Average Pull Request age
  • Average Issue age
  • Number of External commits

#4: Invest in tools

Mostly internally, to help teams run projects. These are internal dashboards, visible by everyone at company. Everyone is aware of the metrics we follow internally. "Big" views on all projects, which ones are doing well/badly, and you can drill down and see the owner for each project. They are clearly defined, an employee, and can assign tasks to directly. I can hassle them, and also, if they ever leave (as it did with Tornado) we can find new stewards for the project. For tornado, we transfered ownership to the community. We have engineers associate their Facebook profile with Github profile via oAuth. We can then track who contributes, whether internal or external. This workflow unlocked so much valuable data about what is going on.

#5: Establish ownership

Don't let projects be orphaned, or flap in the wind. We can show graphs/metrics scoped to projects or teams. Individual teams set quarterly/semesterly goals for themselves often. That social pressure helps projects do well.

#6: Gamification of good behavior

We have teams competing now. React and the iOS pop project have about the same number of followers, and there is a bit of a space-race to get the most followers. In the absence of managing projects directly, you can influence projects. We don't want engineers spinning wheels with lawyers, wasting time. We want them to do it with discipline.

  • How core to Facebook is this technology?
  • Who will use it, who is it useful to, how valuable is it?
  • What else already exists, that is similar to this technology?
  • Is there anything novel in the project?
  • Does it include third party, including third-party open source?
  • Who will maintain the project, accept contributions, and liaise with community?
  • Where/how should project be distributed?
  • What is public release date?

We have a very strong template for licensing, we stick to BSD, occasionally Apache, or Boost, and the only reason we'd look at other licenses, is when target community has a strong culture of using that license. We don't impose licenses unfamiliar to a community.

#6.5: Choose your lawyers wisely

We have a linter to make sure license headers are all there, and everything is good to go, in a private repository. Then we release mid-week, tweet about it, do a Facebook open source social media blast, and then post it to the code blog. Then the social media magic takes holds, and we get good momentum on the first day. We have an internal group of 600-700 employees interested in FOSS.

Every Friday, Mark gathers everyone at 4pm for Q&A. At the start of the session, Mark talks about new apps/products/releases. He's taken to announcing our OSS projects in these meetings, and you can only imagine how motivating that is. Knowing the CEO is aware of a project, and announces it to whole company. Much comes from Infrastructure teams, and that is a huge boost for them. I get a huge surge in interest after Mark talks about them.

#7: Launch is only step zero

You have to know how to continue keeping it successful. I look at the number of followers over time. We can see the bumps of interest over the first week, and a gradual slope over time. It is the gradient of the second half, not just what happens on the fist day.

Some exceptional cases:

  • fb-flo and origami beat this curve; flo was released at a JavaScript conference, tripled their community; face-to-face PR hugely grows FOSS success
  • KVO Controller did two week intervals and saw strong growth after each session; practice makes perfect
  • our climax was the release of Pop, which blew everything away; got 4,000 followers on the first day, 6,000 in the first week, and is way north of 7,000 now

Obviously we benefit with reputation, but the success was built on the success of previous iOS projects. Pop had a closed beta for two weeks before launch. Out of the gate, we had a strong pick-up. Our closed betas were best advocates, helped early growth. The reaction from the iOS community was strong.

We encourage major projects to have their own website. Our design teams have built entire sites for Origami. It shows you care, and take care of your project.

We have IRC, Facebook groups/pages, meetups, and hackathons. It all is important; and it all works.

We have one technique, called a community round-up. The React.js team will gather all the mentions, all the projects, all the demos/presentations, and then shows them to the rest of the community, not just at Facebook. This gives extra authenticity.

The first couple weeks of external commits are vital! In the first day, you'll get a swath of PR'S, most will be typo fixes in documentation. This is not a bug, it shows that people are feeling comfortable.

#8: Leave breadcrumbs

Docs, unimplemented features, to-dos. As projects go on, they change their destiny. There are many paths: Snapshot, Upstream, Flythenest, Deprecate, Reboot.

Snapshots: usually read-only, academic exercises; many are created to get upstream FBThrift is a good example of this

Upstream: we teamed up with Twitter and Linkedin to get changes upstream in WebscaleSQL

Flythenest: project goes on to become "it's own thing;" some of our major projects will have this, and then we'll eventually become "just be a user" like everyone else

deprecate: project served useful purpose, and finishes

Reboot: project starts over again

#9: Understand OSS project lifecycles

We launched 65 new projects in the last couple months. That's about 2.5 projects per week. It is more about quality than quantity, but each has a goal. The is a variety of types of projects; mobile, infrastructure, and programming languages. All are very broad.

MetricsJune 2013July 2014Total Repos129202Followers50.1K97.6KForks11.8K20.7KPull-requests1400 (502 days)1973 (208 days)Issues404 (323 days)427 (186 days)Commits30.7K42.4K

#10: Be open and connected

It has been a pleasure to share our journey with you today.

Q: In the Facebook license, it looked like "for more information."

A: Straightforward BSD license, and a patent grant. We have a patent grant for the developers, same as what happens in the Apache License.

Q: Does Facebook have a Contributor's License Agreement (CLA)?

A: We didn't have slide for the CLA, but it is basically the Apache CLA. It is so users that contributions that came from external contributors were theirs to give. We then have a bot that comes around to do a Github auth. Exactly the same as the Google/Apache process.

Q: Have we open sourced the GitHub scripts?

A: I knew someone was going to ask that! We'll share as much of that as we can soon.

What is your Background?

Name:James PearceTitle:Open Source Program LeadTwitter:@jamespearceLink:code.facebook.com/projects

Been in the tech industry for years, mostly in mobile. Worked on early early mobile tech, when it was called "WAP." I've been waiting for it to become the next big thing, and it finally has. I joined Facebook about three years ago, working on Mobile Developer relations, talking about app integration.

When it came to open source software, it was serendipity. We saw it needed love, and here I am. I'm still learning a lot as I go along. We try to federate as much activity as we can, and make it as light touch as possible. We're doing better than we were, but we've got a long way to go. We've got lots of projects, but we want to do more, work with more communities, and think more about how we provide stewardship over time.

How do we do more in mobile? We have lots to offer in Android, and we want to continue to run the program as efficiently as possible.

How can people get involved?

Check out our careers site. All our open source projects are on GitHub, we're friendly, and we're responsive when people send pull-requests.

This derivative work by Remy Decausemaker is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Lead Image: 
Neon sign: Internet
Rating: 
(7 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

Code.org on reaching the next 100 million computer scientists (SIGCSE keynote)

The following is an adapted transcription from the keynote address given at the 2014 SIGCSE conference by Hadi Partovi, founder of Code.org.


 

Thanks for the warm welcome!

Last year, SIGCSE (Special Interested Group on Computer Science Education) was a week after our launch. It questioned our motives, and existence. We made a video, and that that video got 12 million views, so I built an organization around it.

This year, I was welcomed very warmly. I was up until 3 AM last night, which helped remind me that Computer Science educators are not only the coolest, but also the most innovative.

For those who aren't familiar with Code.org, there are lots of opinions of what we are. Lots of people think of us as a marketing org that makes videos of celebs, a coalition of tech folks filling Computer Science jobs, a political advocacy organization for educators and technologists, the organizer of the Hour of Code, a software engineering house, a curriculum writing team, and a grassroots movement to bring together for-profit, non-profit, and Governmental organizations on a united goal.

Our vision: Computer Science taught in every school to every student. Not required necessarily, but definitely the opportunity to take Computer Science available to all.

Three things we talk about

  1. Bridging the student/job gap. Politicians think about this a lot
  2. Reaching underrepresented students, and the social justice implied in doing so
  3. Computer Science is foundational for the 21st century. Doctors, lawyers, even the President of United States, need to know about this field

As early as 15 years ago, I asked our Dean why don't we do these things, he said "give it 15 years, it will fix it self." But it didn't...

The myths

  1. We're all hype and only do Hour of Code. In fact, we have 15 staff at this event alone and we're always working.
  2. We want to do everything by ourselves. In reality, we have over 100 partners who help us do everything we do.
  3. We are only about "coding" or learning to code. This is probably due to the name mostly, but if we called ourselves "computer science for the masses," our URL would be csftm.org, which is just as confusing as any other acronym.
  4. People assume Code.org is about the software industry coming to tell schools how to do their jobs. I have a Computer Science background, and I've gotten software companies to fund Code.org, but that doesn't mean they run it. Much of our money is spent in Kindergarten on kids who will never become computer scientists.

The difference between computational thinking versus programming for CS is clear to all of us, but for the average person it may be jumbled. We don't just want people to learn to code, we want people to learn to think. We are disrupting things—not the natural order, but the previous order. We want to disrupt education in a goodway.

The pillars of Code.org

  1. Educate: Bring CS to all K12 schools in US. This is the biggest job we're trying to tackle. Making curriculum, and working within school districts.
  2. Advocate: Remove legislative barriers, make CS part of core academic standards.
  3. Celebrate: Combat stereotypes that prevent more students from joining in CS.

Hour of Code highlights

  • 28 Million students, in 35,000 classrooms
  • 48% were girls! (huge applause from audience)
  • 30 languages, 170 countries: many volunteers helped translate
  • Insanely high ratings (97% positive vs .2% negative). This was our first time doing something "real" in public schools, and this rating came from teachers. 75% gave it a 5 and 22% gave it a 4.
  • 20-hour K-8 introductory course: 800,000 students are participating in 13,000 classrooms; 40% girls!
  • School district partnerships: 23 districts, including the #1, #3, and #6 in the US. We've held professional Development workshops this summer for ~500 teachers from K12.
  • State advocacy: we changed policy in five states, with eight more on deck!
  • A lean team: we hired eight full-time staff

How did we do all this? Partnerships across industry, non-profit, government:

Pillar #1: Educate

Code.org has a developed a full K-12 curriculum:

  • 20 hour modules all the way to middle school
  • Aligned to Common Core standards in math and ELA
  • Middle school modules go into Math and Science classes. Teaching Math science via Computer Science
  • High school intro course, and a high school AP course

Code.org has two different models for how we spread.

First is the Online Model, where we're focused on putting more courses and curriculum online for teachers and students. The lower the grade, the more freedom teachers seem to have. 3rd grade teachers teach math, sure, but they are not just a math teacher, and can find ways to integrate code into more activities. This is extremely cost effective, about $0.05/student!

Then there's the District Model, where the district provides teachers, classrooms, computers we provide: stipend, curriculum, and marketing. This helps make sure there is no cost to the school for adopting a Computer Science curriculum. Managing costs for scale: around $5K-10K/high school, $5,000 x 20,000 high schools who don't have CS; that adds up.

Code.org is looking into developing things like:

  • State level Teacher Certification Exams
  • Incentives/scholarships for studying CS in Schools of Education.
  • Building a pipeline of pre-service teachers

We've found the Holy Grail for online curriculum is to make learning feel like a game. An online curriculum makes teacher's lives easier. This is not about making an "end-run" at teachers; Web-based curriculum reduces the IT hassle Significantly! Most high school CS teachers in this room, also double as the de facto IT person in the school!

(audible "yes" from many and laughter heard around the room)

As long as the IT Department doesn't blacklist us, you can get to our IDE and curriculum. We have a team of engineers working together to blend curriculum and game design. We're still early on in evaluating the results. In the web-world, you run the data through Hadoop and/or Hive, and we've got 10M datapoints.

Some ways people can help

Universities:

  • Bring a Computer Science Principles course to your institution
  • Partner with your School of Education to bring more Computer Science into the Ed program - ideally a teaching-methods course, or any Computer Science Endorsement.
  • Give support/instruction to the "tech ed" courses at local high schools.
  • Help Code.org scale by offering K-5 workshops for us! Email univ@code.org if interested.

Anybody:

  • Convince your local school district to teach CS. (Code.org will enter new regions if 30+ high schools are on board)

  • Help us improve our curriculum (Code.org/hints). The Hour of code is behind us now, but we're still getting 1M students to the site every week!

Pillar #2: Advocate

Computer Science is foundational! Every student should have access. Computer Science should be core academic offering in school, not just a vocational elective on the side. Code.org takes a broad approach. We make recommendations for states to adopt. For further reading see The ACM Report Rebooting Pathways to Success.

At the national level, we have the Computer Science Education Act, which has bi-partisan sponsors in both houses. It says more or less that STEM funding can be used for Computer Science. It's a highly non-controversial bill. Small amount of optimism that it will be passed, but since this is the most unproductive congress ever...

At the state level, we want schools to allow Computer Science to satisfy existing high school math/science graduation requirements. At the university level we want to make Computer Science count. We want Computer Science to satisfy math/science College admissions requirements. We need universities to recognize the above point.

Where it counts

  • CS enrollment is 50% higher in states where it counts
  • 37% more participation by African American and Hispanic students
  • Calculus enrollment remained unchanged
  • We have legislation on deck for states like: FL, NY, IL, CA, AZ, OK OK
  • We have policy recommendations in the works on a district level in states like: WA, KY, MI, CO, MA

We're going to start a collaborative whitepaper for universities to accept AP/IB Computer Science to satisfy math/science requirements.

Pillar #3: Celebrate

Hour of Code is a hard act to follow. Fastest web technology to reach 15 million users! It took Tumblr 3 years, and Instagram 14 months. We did it in 5 days. 

More girls participated in US schools in Computer Science Education Week in seven days, than the last 70 Years.

(huge applause from SIGCSE audience)

The 2014 Computer Science Education Week is December 8-14. Our goal is that 100 million students try The Hour of Code in 2014. It will require participation by a majority of US students, plus broad international participation. You can help by asking your school to participate, and by buying and wearing Code.org swag.

Closing thoughts

Computer Science is at an incredible inflection point. There are people here doing what I've been doing for 10 times longer. If you've tried before and failed, try again. It wasn't easy to get the President to talk about Computer Science, but it was easier than ever before. Leverage the numbers that are now possible.

With shared goals, anything is possible.

After his keynote address, Hadi generously obliged the author with a one-on-one follow-up interview. Below are his responses.

Where are you from?

I'm originally form Tehran, Iran, and now from Bellevue Washington. Code.org is based in Seattle, Washington.

Where did you study?

Harvard grad, BS/MS in Computer Science, 1994.

Any clubs/activities outside of school?

Our computer programming team, took 7th in the world in the ACM International Collegiate Programming Competition my senior year.

Why did you start Code.org?

I think the fundamental reason, is it should happen. Anyone who tries Computer Science and programming, a lightbulb goes on. It teaches creativity and it is powerful. It seems un-American that 90% of schools don't offer Computer Science. I'm living the dream, sure, but it is a dream that 90% of kids won't have access to.

Fewer schools teach Computer Science now than 10 years ago. This lack of Computer Science is breaking the American Dream.

The very seed of Code.org was a technology roundtable in December of 2011 with President Obama. As I was listening to myself speak, I thought "no one is going to do something about this..." In March of 2012, I was at a conference with Jack Dorsey and Drew Houston. I talked about making a video, which started as a hobby idea, but as soon as they said "yes" I've been on a path that hasn't relented since.

What are your thoughts on Free/Open Source Software?

Personally I firmly believe that education should be as free and open as possible. The intellectual property around education should be both free and open source. All the curriculum we create will be licensed Creative Commons, and all the code open source. We want the community and volunteers to help us. We get asked all the time "Can you do this? Can you do that?" and the best answer is "Here is the source code, go ahead and do it."

We've made tutorials where we have Angry Birds and pigs, and countries tell us that that is a religious issue. When it is open source, we tell them to change it, and they can.

We have prisons that want to use our stuff, but can't have Internet, so we want them to adapt offline versions of our code.

Those are things that we didn't even imagine when we started, and can happen through open source.

Final thoughts?

Even if you are not an engineer, it takes five minutes to see how fun the tutorials are. Even as an adult, you can learn to code too!

If you want to get involved, http://Code.org/help is the place for people who want to help us.


 

This derivative work by Remy Decausemaker is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

 

Lead Image: 
code.org keynote address
Rating: 
(5 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

Hacking computer science education at Khan Academy

The following literary transduction is based on a lecture by John Resig given at the Rochester Institute of Technology's (RIT) Center for Media, Arts, Games, Interaction, and Creativity (MAGIC).

I'm deeply honored to be here today. I keep coming back to RIT, every couple of years, and I live in New York City now. There are always new buildings and new things happening. Very fantastic.

I'm here to talk about the stuff I'm doing with Khan Academy. I'm the Dean of Computer Science, which is really a very tongue-in-cheek thing since we can make up our titles. I guess, I'd start with an intro to what Khan Academy is and dig deeper into what I'm doing with CS (computer science) platform in particular.

I joined Khan Academy in 2011 after I worked at Mozilla doing JavaScript tool development. The big thing, our goal, is: bring world-class education to everyone, everywhere, for freeWe are really good at producing educational material. Our math, science, computer science, and art history are very good. Most content is released under a Creative Commons license, so you can take it and do what you want with it.

We're making inroads to bring our content to the world. I've been working on the internationalization of Khan Academy. There are some problems in computer science that are "solved problems," and you'd think doing a website in multiple languages would have been one of these... but it is hard. So many edge-cases... We have videos, articles, exercises, all things relating to our curriculum.

Who here uses Khan Academy's math curriculum as a supplement? (many people raise hands) That's pretty good! We usually tend to skew younger than college age!

Tracking better data

One of the things we do is track all the work you're doing, so you can get a more effective education. Every exercise and video, we track when you did it and how it went. If you got right answers or wrong answers; we can figure out why you're getting it wrong and how to make it better. We have a dashboard where we can push you to work on exercises to move your knowledge forward.

This was the first thing I built at Khan Academy, this framework for easily creating exercises. This is a system where you have a problem, charts, hints, and when you answer stuff, it can steer you when you get things wrong, to give you a better understanding. I worked on the internationalization for this as well, and it was ridiculously hard. Exercises made a lot of assumptions about English and western ways of doing this.

Take "Jane gives one ball to Fred," for example. "Oh, that's easy, we'll just replace the words!" But every language has differing ideas of what is singular and what is plural. Some places have more or less rules and even separate rules if the number ends with a three! We are now available in Spanish and Brazilian Portuguese.

Much of our material on the site is split into a tutorial format. This is where you can go through, watch videos, and do exercises in a very linear form. This is great for our math content, and you see many students at middle-school and grade-school level using this. As I said before, we're tracking lots of data to give students a better experience, but also, the teachers.

Khan Academy is trying to change how we think about education. So we can provide teachers with insights on how students are understanding content and progressing. You can see how they are struggling.

We want to change the "traditional" model where you go in and the teacher gives a lecture to students about a material and assumes that everyone is at the same level. It assumes everyone has equal footing, and often, this is not the case. You have students lagging, and students who are wayyyy ahead. If we had better data, teachers could make more informed decisions.

One-to-one teaching

We break it down for the teacher so they can track specifically what students are working on at a given time, and analyze students in a class or across classes. No longer do they need to give a general lecture. Now it's "Ok, it's math time. You set your goals."

This week a student decides they want to finish long division. They can work on that and watch videos. The teacher can track whether they are getting stuck, and then give a targeted lecture. If there are four students struggling with long division, the teacher can give those four a lecture to reinforce those concepts, rather than doing the same thing over and over again with the entire class. This means they have time to do more one-to-one teaching. They can see who has higher or lower levels of understanding. This is something that provides a huge level of insight to teachers to better use their time.

We have lots of students using this system now: over 3 million problems per day! There are huge spikes in the beginning of school in September—we hit 10 million active students. I'd like to teach on a classroom basis, sure, but this is a way that I can better expand and stretch myself to teach computer science content. Depending how you measure, we have 250,000 to 1 million students doing computer science stuff on the site.

Computer science at Khan Academy 

I started working on the computer science platform in late 2011 and released it in August 2012. We iterated a number of times and settled on this final implementation. The big thing here: this is on the computer science part of the website. We have a lot of curriculum. My coworker, Pamela Fox, is producing all the content (videos exercises, etc.).

We go up to object-oriented techniques, not much overlap with Computer Science 101, and we have a long way to go to be a "full" computer science curriculum. You probably won't be able to go out from here and get a job yet. I want to encourage people, expecially at the target of middle-school and up, in finding the thing that excites them most about programming, and applying it to their life.

Showing people the ways that programming can help improve science, biology, or art. I'd love it if our computer science platform didn't jut produce computer science students, but enabled students to do what they love, with programming to help them do it well. Art historians who can program. Humanities people who can code. Cross-pollination for disciplines that traditionally don't overlap. This is hard, but this is what I'd like to see.

Coding for all disciplines

Who is familiar with GitHub(many hands raise) You can edit code, fork it, upload it, and make changes to the original. Similarly, what we have on Khan Academy, is the ability to make programs. Has anyone heard of Flappy Bird? Someone rewrote this within Khan Academy!

We don't allow outside images yet, so anything you can see here on the computer science site is drawnSo, someone made this game, a student I don't know. It got 700+ votes so far and 1600+ "forks," which we call spin-offs. These are all variations on what has been made. It looks like most of them are about the same. Slight changes in the color of the "flappy bird," stuff like that. Students are taking the code and modifying it. This program has already had over 1000 spin-offs, or programs which were cloned or modified in some way. The top spin-off itself has 113 votes, and over 100 of it's own spin-offs! This is a very different collaborative environment. People making something and learning from it.

Khan's collaborative environment

I wanted to build off of the open source model. I wanted the code to be the front and center and not just show the graphical content, even when it is not a programming exercise.

We have a partnership with NASA, and we're doing lots of simulation stuff. One is making a lander go into orbit and land. It is really hard. Though it isn't required, I wanted to show the code. People who are interested in space can look at the simulation, see it, and say "Hey, how does this work? How can I learn more?" I can go in and modify the simulation, save it as my own spin-off, and make whatever changes I want. That is the model we've embraced here.

Stuff like Minecraft and Cut The Rope style games, those are always popular here. Someone made a drawing program. You can't really save "state," so in the program, at the end, it spits out a giant blob of text, which is what you can copy-and-paste into a program to recreate your drawing!

Students adapt, modify, and change

Students have found ways to work around our system. This type of "meta" programming and activity is what it is all about. Students are even making clubs now. They will hang out and chat with each other in the comments section. Social groups are forming.

Students also tend to make lots of demands and often petition for lots of things. Usually you'll find one at the top of just about every page. One thing often requested is support for playing sound, which I suspect students will abuse plenty. (audience has a good laugh) I like when students can discover things for themselves. What they learn from, it tends to be much more deeply embedded. They take more pride in it.

This has been my baby, real-time injection of JavaScript code, so you can inject "state" into live running code. I feel like it is worth it. You get a much more compelling experience, and you can manipulate things as they happen. You don't have to wait for the program to restart. Even as we we're typing in the code, the program was still running! It works pretty well.

We can record all the actions, and keystrokes, and things being said. We can play back commands and audio. We use videos frequently, but I don't think they are great for computer programming. The thing you want students to do is take some code, pause it, and change things to figure it out for themselves.

We had some really janky solutions where we would pause YouTube videos and regenerate code, but we got to this point. Now, you can pause, make a change, and then when you hit play, it reverts the code, and continues from where you left off.

We also have interactive transcripts now too, so that if you are hard of hearing, or you want the transcript for what has been said, you can get it. And we are now starting to translate those into other languages too (we're working on translating the audio).

All the ability to do this stuff is powered by browser technology, like HTML5. We pull content from Amazon S3, and it works surprisingly well.

I showed you math exercises where a student is given a math problem, they put in an answer, and then they get a "right or wrong" result. That works OK for math, but not really for programming. What works well for learning program is actually writing code. I want the students to be writing out the code. This is one step above recordings. We are basically showing you what code to write, and based on that, they can turn it into their own thing. There are much more complicated programs that can be made from the basics.

One thing I'll mention is that a lot of code is open source and available on GitHub. If you are interested in contributing back, that is all there.

We built a framework this summer called Structured.js. We think it's really cool. You can define a rough structure for the code you want students to write, and then analyze the code to see if it matches a structure. We parse the student code, convert into a syntax tree, and compare it to our syntax tree.

We use jQuery on the front-end, along with backbone.js and a new framework from Facebook called react. It is pretty crazy, and I think you should check it out. I'm slowly getting used to it. On the back-end, we use Python on Google's App Engine. Sometimes we get on 60 Minutes, and the traffic goes bonkers for an hour or two, then it goes back down. Google App Engine handles it quite well.

I think I will wrap up there and answer any questions anyone might have. Questions about what I'm doing at Khan Academy, questions about jQuery of course. (audience laughter)

Q & A session

Q: Does the order in which things are written matter in Structured.js?

A: In this specific case here, it is forcing you to do your 'if' statement before the loop, so yes. A tool like this could be used poorly, sure. And that is why we have such detailed hints. We provide the structure itself as a hint. This is something here, that is one step above a video. They can name their variables whatever they want, we don't care, which is an improvement from most systems like this. This is relatively new, but students love it, and they are using it a lot.

Q: You are doing this teaching in the browser, which is good, but are there any "where to go next?" resources on Khan Academy?

A: We have an article called "what to learn next" where we direct students to tutorials and articles, projects to work on, web development in general, and other language resources. At this point, we are not everything for everyone, but we want to be the launching pad. Here is your first taste, and then you push off from there.

Q: The structure was all S3 and Google App Engine?

A: Yes, all cloud hosted. S3 for file hosting, videos typically pulled from YouTube. We don't have any physical servers. That is the nice thing about being "in the future" now, we don't need any of that. (audience laughter)

Q: What is the history of the name?

A: It was created by Salman Khan. He created many of the videos on the site. We also have other professors and professionals who produce content, but he has done most of it. It started as his YouTube channel to help his cousin learn math. It got bigger and bigger, until it's it got up to like a million people.

Q: How do you moderate all the content? There could be potentially negative things, right?

A: We have a system for people to "flag" content, which puts it into a moderators queue. Once it gets three flags, it gets automatically moderated, and then can be reapproved by a moderator. We haven't had anything too terrible show up yet. It's been running since 2012. Really, it's not the programs, as much as the comments... BIG SURPRISE! (audience laughter) There have been "factions" of middle-school students who battle in the comments sections. We were worried in the beginning, but this hasn't actually happened yet. We banned outside images and only allow images that we provide. We had a student write a program that turns an image into a multi-lined call to the rectangle function to recreate it. If we find a way to ban something, students will always find a way around it. They are mostly doing things like putting up pictures of their favorite Pokémon.

Q: Biggest tech challenge at Khan Academy?

A: The real-time stuff was pretty huge. The internationalization stuff of the past year was really hard, but hard in a different way. Not hard in the "how do we scale" way, but in the "understanding the cross-cultural issues" way. One of the hard problems for his platform, was making it in such a way that young students can understand what is going on. We ran a whole summer school and did lots of playtesting. We'd get a different batch every week. They'd get summer school credit, and we'd get bug reports. We were in there with our notepads, and running everything through JSHint. We were trying to provide errors that were much more intuitive, but this wasn't explicitly a tech problem.

Q: You talked about "clubs" forming in the comments, I was wondering how far you were willing to take the pseudo-social aspect? Are there plans to expend into different languages or other types of programming?

A: Yes, in some areas. One immediately I think of is humanities. I didn't think about this initially, but the way in which you are making these programs and building a creative work, you spend lots of time and you want people to collaborate. This can happen for writing, poetry, art, music, all of these things for which there isn't a model for teaching online. Having the staffs and bars for composing music and hearing the playback in real-time would be great. Even better would be for live-forking and editing for others to collaborate. More immediate feedback on what projects students build is a goal.

The only other language we're looking to go into is HTML/CSS/JavaScript. The huge advantage of JavaScript is that it can run natively, you don't need to pass it back and forth. For now, we'll be sticking with browser native languages.

Q: Khan Academy is free, right? How are you making money right now, and do you see any change in the future?

A: We are funded by grants. The Bill and Melinda Gates Foundation, The Carlos Slim Foundation, and others. There is a lot of interest, and funding, and we aren't in the position of every other education startup—where you're charging teachers, schools, or students money. I don't want to be pinching pennies out of students pockets. For additional revenue, we have contracts for folks who are using our content in commercial contexts.

We do have lots of jobs, tons of internships. The whole computer science platform was built by me and interns, so if you wanna work with me and build cool stuff, please do.

 


This article is based on work at http://ejohn.org.

 

Lead Image: 
Rating: 
(9 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: 

Wrapping up the Summer of Code at the Googleplex

Over 280 attendees representing 177 mentoring organizations gathered for a two-day, code-munity extravaganza celebrating the conclusion of Google Summer of Code with the annual Mentor Summit held at Google in Mountain View, California.

Mentors and admins began arriving on Friday night, and walking about you could catch bits of conversation, spoken in a plethora of languages and accents, spanning from pixels to bits. No less than four trips of double-decker bus loads, from two different hotels, shuttled everyone into the Googleplex. The morning began with a hearty breakfast and coffee from Google's expert baristas. With trays piled high with eggs, bacon, muffins, and other breakfast-y goodness, mentors took their seats in the massive company cafeteria. Under a quartet of stage lights in that familiar Google colored glow, Google Summer of Code lead Carol Smith stepped up to the microphone, and welcomed the crowd.

Once folks were acquainted with the schedule of events, places of interest, and policies to follow, Free Open Source Software (FOSS) advocate and Director of Open Source Programs at Google, Chris DiBona, addressed the audience:

The reason you are here is because you deserve to be. The whole point of GSoC is to introduce new developers to FOSS, create more FOSS code, and support projects we think are great. We look at reviews, and the aftermath and say 'did it work?' You are here because it did. Thank you for being there for open source software. Thank you for being there for free software, and for being there for Google. Open source matters to us. The future of open source matters to us. This room, and the people you bring in, without you, it wouldn't be as wonderful in 5-10 years as it is today.

The big reveal

Prior to the summit unconference, attendees had a chance to suggest and vote on session topics using Google moderator. Sessions were assigned to rooms of a size proportionate to their level of interest. Ample space was also provided for sessions that were proposed on-the-spot, often inspired by discussions from previous sessions.

The Pumphandle session

Pumphandle session

Photo by Thomas Bonte

The first session of the unconference took the entire GSoC audience, split it down the middle, and formed two long lines for a full morning of meet-and-greet handshaking. This provided attendees with an opportunity to meet each other and have conversations they may not have had otherwise during the busy summit.

The Chocolate Room

Behold, the annual cocoa cornucopia! Mentors from around the world packed  plenty of sweet treats to share with their fellow hackers. Milk chocolate, dark chocolate, bacon chocolate, and yes, even fish chocolate.

The GSoC Band

GSoC Band

Photo by Thomas Bonte

In the open source community, ad hoc collaborative teams are an everyday occurrence. But to see it happen outside of a source code repository, with a full drum set, five kinds of stringed instruments, a keyboard, and even an oboe... that is something you don't see everyday. Shout out to Saturday night's Emcee, host and bringer of instruments, Googler Marty Conner, who got the GSoC band back together for 2013.

The Sticker Swap

Over the course of the summit, Googlers would freshen the tables of swag at the front of the cafeteria. Tshirts, banners, stickers, and even GSoC socks! But Google wasn't the only team with a horse in the swag race. Mentors brought stacks of stickers from dozens of projects to participate in the annual sticker swap.

Googleplex tours

During the lunch hour each day, Stephanie Taylor and Mary Radomile of Google's Open Source Programs Office gave attendees guided tours of the Googleplex campus.

 With each new release of Google's Android operating system comes a new codename and a new statue in the Sculpture Garden. Note the new KitKat Android on the right side of the photo.

The Cakes

Thanks to Joel Sherrill with RTEMS, who supplied the templates for the giant Google Summer of Code birthday cakes, celebrating nine consecutive years of FOSS community engagement with the logos for each year of the program on two tasty cakes.

A new GSoC tradition

Based on feedback from last year's summit, the organizers agreed to put together a whole track of Google talks, given by current employees about a variety of projects, initiatives, and technologies. One of the more popular sessions was led by Wesley Chun, Developer Advocate with the Google Cloud Team. Chun talked about the Google Cloud Platform, its variety of services, and special discounts and support provided by Google to FOSS projects.

Big takeaways

GSoC Mentor Summit

Photo by Matthew Dillon

As a first-time Google Summer of Code Mentor, attending my first summit, I cannot even begin to recount all of the amazing things that occurred over the course of the weekend. If you clicked on the link at the top of this article for the 177 mentoring organizations represented at the summit, you can begin to imagine the sheer magnitude of talent, passion, and dedication that gathered in Mountain View.

As a storyteller, I accumulated thousands of words worth of notes from all the sessions I attended, which sadly, I cannot possibly share with all of you readers in a single post, so we're going to have to do a highlight reel of sorts.

Operating Systems Summit: When else do you see core developers from Gentoo, Debian, Fedora, NetBSD, FreeBSD, DragonFlyBSD, and others, all politicking in one place?

Gamification in FOSS session: Tales of developer incentivization were shared by projects such as Joomla, Battle For Wesnoth, and the Fedora Community.

Humanitarian Free Open Source Software (HFOSS) session: HFOSS.org founders and members, met with representatives from other projects such as OpenMRS, Sigmah, PostgreSQL, The Sahana Software Foundation, The Tsunami Information Project, Mifos, NetBSD, SugarLabs, BRL-CAD, and a handful of others, to discuss our role as hackers to improve the conditions of our planet, and our species.

Outreach Program For Women: Led by Karen Sandler, Executive Director of the Gnome Foundation, who introduced the OPW, and discussed ways to bring more diversity to your FOSS project.

Next year will mark the 10th year of Google Summer of Code! In honor of the-big-one-oh, Google will be expanding the Google Summer of Code program 10% across the board:

  • 10% increase in student stipend
  • 10% increase in total number of students accepted
  • 10% more accepted Mentor organizations

  


 

Like what you see here? Is your project interested in mentoring? Are you a student that wants to get paid to work on free and/or open source software with world-class hackers? Then you should apply for Google Summer of Code 2014. See the original article for a list of important dates!

Originally posted on Google Summer of Code blog. Reposted using Creative Commons.


Lead Image: 
Google Summer of Code annual Mentor Summit
Rating: 
(7 votes)
Add This: 
Channel: 
Article Type: 
Default CC License: