Auditing and Improving Your JavaScript Code - Lately in JavaScript podcast episode 37 - JS Classes blog

Show notes

Download Size: 55MB Listeners: 1265

Introduction music: Riviera by Ernani Joppert, São Paulo, Brazil

RSS 2.0 feed compliant with iTunes:

http://www.jsclasses.org/blog/category/podcast/post/latest.rss

Watch the podcast video

Note that the timestamps below in the transcript may not match the same positions in the video because they were based on the audio timestamps and the audio was compacted to truncate silence periods.

Show notes

• Accessing Amazon API directly from the browser with JavaScript

• The Unnofficial JavaScript Gmail API

• Identify Vulnerable JavaScript Libraries with Retire.js

• Generate Custom JavaScript Coding Guidelines with JSCode

• Extensible JavaScript Code Auditing with ESLint

• Refactoring JavaScript Code with GraspJS

• Installing JavaScript packages with Composer PHP

Read the podcast transcript

Introduction (00:20)

[Music]

Manuel Lemos: Hello. Welcome to the Lately in JavaScript podcast  hangout. This time again I have here with Arturs Sosins, directly from Latvia. Riga right?

Arturs Sosins: No, but very near. Hello from snowy Latvia. We already have snow here and it's kind of getting cold.

Manuel Lemos: Yeah, that's great. Do you ski? No?

Arturs Sosins: No. I build snowmen.

Manuel Lemos: OK, at least it's related with the cold. Over here in Brazil, it's almost summer. And it's always hot over here this time and rainy. Today is not raining, but the other days it had been raining a lot.

Accessing Amazon API directly from the browser with JavaScript (1:10)

Manuel Lemos: Anyway, we are here to talk about JavaScript, what happened in the latest times, latest weeks. And we are ready to get started with several interesting topics.

This time, I would like to start talking about, let me share the screen here, an API for Web services. In this case, it's from Amazon.

Amazon always have many APIs but they were mostly for use from server-side languages like PHP and others. And this time, they are publishing a preview of their SDK for JavaScript.

So, what this seems to mean from what I could understand is that from now on, you will be able to call their APIs, their Web services directly from the browser, so we don't need a server-side language to actually call their Web services.

They even say explicitly "JavaScript in the browser", not JavaScript from, for instance, Node.js on the server.

Arturs, did you check this library? Did you try it or at least looked at it somehow?

Arturs Sosins: Yeah, actually, I was quite surprised when you presented this topic because it's actually quite great news.

Recently, I have published in JS Classes one kind of Web app package that was Web Notes. And many users were requesting about adding a feature to synchronize these nodes between browsers' devices, and actually providing the plugin to such a service could a great option to do that. So I would definitely check it out more.

The only service that provided something like that previously, I think, was Firebase. Have you heard of it?

Manuel Lemos: Firebase? Firebase, isn't that a database?

Arturs Sosins: Well, yeah. Basically, they have a server and they provide JavaScript to API to access it. Again, you could do something similarly. You could access it and store information from the browser, yeah.

Manuel Lemos: You mean connect to the database from the browser or something like that?

Arturs Sosins: It's basically like what's Amazon is offering to simply store  key values and something like that.

Manuel Lemos: Right. Well, this is interesting. Actually, it's not exactly new that you can send an AJAX request to Web services because it's all HTTP requests.

And what is interesting is that this time, Amazon is making that an official SDK for doing that from the browser. It's no longer somebody trying to...

Arturs Sosins: Hijack their way?

Manuel Lemos: Because it's HTTP and anybody could develop that library. What I was wondering is if you... I'm not sure how this API works.

Probably this is not OAuth-based. But when you use OAuth, there's an exchange of tokens and you actually need to be on a browser.

And there's many APIs for some type like, for instance, Facebook. They can connect to their APIs via JavaScript. There is an SDK, too. But it is OAuth-based, so it means, in the end, they have a token so can send request to the API on behalf of the user.

My concern is, I don't know if it is the case of Amazon. If there are any password, any tokens, where are those stored? Isn't that something that could be somehow abused, made easy to steal those values? I don't know if they have thought of that.

Arturs Sosins: There is a section about authentication access control. And basically, they allow you to authenticate with the same Facebook SDK that also is written in JavaScript, or even Google or even using your own Amazon login information. So, they have thought about it.

Manuel Lemos: Well, I confess I did not have the time to look into this, but...

Arturs Sosins: You can even use it from mobile devices. And they're saying that they can create a temporary security credentials just for mobile device authenticate and then use it.

Manuel Lemos: Yeah, but where are those credentials stored? Are they in local storage or cookies?

Arturs Sosins: I don't know. We need to jump in and read a little more.

Manuel Lemos: Yes, I also have that question so I was wondering.

You mentioned you have your package that you're considering using this API or whatever to probably store data in S3, right?

Arturs Sosins: Yes.

Manuel Lemos: So in your case, you would need to store the user password somewhere so you don't have to ask the user password or token or whatever it is somewhere.

Arturs Sosins: That could be local storage.

Manuel Lemos: Yeah. Do you think using local storage wouldn't raise some security concerns regarding where the information is stored? It may be stolen by some malicious application or plugin or whatever, Obama, NSA?

[Laughter]

Arturs Sosins: Might be. But just months ago, we reviewed the package that can store the information securely. Again, in that case and when you think about what store the keys and stuff like that, but I don't know of...

For now, local storage seems to be quite secure as it can be accessed from a specific website. And I actually don't know if you can access it normally from a computer side.

Manuel Lemos: Yes, I understand that from the browser. Different domains would not have access to local files of other domains.

Arturs Sosins: Exactly.

Manuel Lemos: So, from that point it would be secure, but I think that information is stored somewhere in the user browser and probably be subject of being stolen.

Although, if you remember, I think it would probably be good idea to also use that library that... I think it was Jason Gerfen that created it to encrypt data in local storage using probably a key that probably does not get stored there. Or else it wouldn't be secured.

Manuel Lemos: Well, anyway, this is subject to more study because we don't know how far this APIs have gone in terms of thinking of all details of possible security exploits.

But for now, it's just a preview, a developer preview, of this Amazon Web services SDK for JavaScript in the browser.

The Unnofficial JavaScript Gmail API (9:10)

Manuel Lemos: And now talking also about APIs available to JavaScript in the browser, I would like to also comment about an interesting API because it seems to be what we thought it could be.

If it is not official, nothing prevents somebody of creating an official API. And in this case, it would be an API for Gmail. As far back as I can remember, there are no official APIs for Gmail for any language, but I confess I did not check this to verify if it's accurate, that statement.

Anyway, what we can see in this page, which is basically a project in GitHub, is an unofficial API from Kartik Talwar.

He seems to have implemented several functions in the API that can access data from the user emails. I'm just not sure how this works in practice, because it seems to even tell what is the current page, what are the numbers, what are the visible emails.

Arturs Sosins: Well, there seems to be two ways to use it. Either you can create a Chrome extension, which would work with what document that page is displaying or you can inject it directly into the page, either through the developer tools or through the address bar and all that. There may be many ways to inject the JavaScript.

Manuel Lemos: Oh, really? So, it wouldn't be actually some JavaScript that you have on some page from another domain?

Arturs Sosins: Yeah. So, very simple. It's more like hacking an API.

Manuel Lemos: Yes. Because it provides quite an interesting set of functions that are not just those types of functions that you would call from server-side API. Because these seem to be accessed information from the user interface, like I said, visible emails, current page.

Arturs Sosins: Yeah, basically, it parses the HTML and then provides the results.

Manuel Lemos: Yes, that's probably what it does because as far as I can remember, Google does not provide any type of API for Gmail.

Now, that you've mentioned development of Chrome extension, that will also be interesting in the sense that if somebody has an unofficial Gmail API and put it in the Chrome extensions AppStore or whatever they call it, would Google approve it? Since it's not official, it's probably a workaround something that is not allowed.

Arturs Sosins: Most probably.

Manuel Lemos: Would Google allow it? I don't know. I know that Google sort of censors some extensions that would allow users to download videos from YouTube. So, they probably were advised to limit what they can do.

Arturs Sosins: If you would search, I think there is no extension that uses this JavaScript, so maybe it's not allowed.

Manuel Lemos: Yeah. And there you are, yet another great API that addresses the needs of many developers but you can't use it.

[Laughter]

Manuel Lemos: Oh, well, that's life.

Identify Vulnerable JavaScript Libraries with Retire.js (13:06)

Manuel Lemos: Well, OK, anyway, moving on now to a different topic. Actually, I have planned to talk about several interesting articles that are related with topics, that are somehow related with similar things or related things. Let me start here by sharing the screen and let me...

OK, this one is, this first one here is Retire.js, which is basically a library for identifying libraries being used somewhere and that have known vulnerabilities.

I also did not have time to try this, but it seems to be probably more useful as a Chrome extension because if you use it as a Chrome extension, you probably can tell which are the JavaScript libraries being used on the current page and then tell if this page is using some vulnerable JavaScript libraries.

And if you are the developer of this page or this site, you should probably update it. That's what I got. But it also has other parts like a command line scanner and a Grunt plugin. These I'm not sure how they would work.

Did you try this, maybe looked into this in some more detail to understand how it works in practice?

Arturs Sosins: Well, command line scanner probably would scan local files, local HTML files, to see what JavaScript packages are used or simply scan folders. And then, if there was detected some kind of vulnerability in the observation, it would say you could update it. Basically, similar to what Chrome extension would do.

Manuel Lemos: Right. But how would you tell that the page... What would be the... It would retrieve the page from the command line and...

Arturs Sosins: Yeah, probably, read the file as a text file and parse for JavaScript.

Manuel Lemos: Oh, I see. Because sometimes, you are using JavaScript libraries that loaded dynamically, so you don't see what are the JavaScript libraries until you are running it on a real browser.

Well, at least, for static pages, I mean pages that are not loading libraries dynamically, it probably can be useful.

Arturs Sosins: Actually, there are multiple kind of points which I can't understand right now how it's done. Basically, how would you determine if a library has a vulnerability simply by checking if it has an update or not? Or how would you check the versions of a specific library?

Not everyone puts something in the code, I don't know, to specify version or something. It should be very library-specific, so how do they actually do it? They probably have some kind of database or something like that.

Yeah, probably they are able to do some fingerprinting to determine the version of the library when it is not that obvious. Well, anyway, I think this is good news, because we don't see many tools for...

Arturs Sosins: Such project management, basically.

Manuel Lemos: I mean for security purposes, auditing or detecting insecure libraries for JavaScript. Well, at least for me, this is the first time I see one of those libraries.

So, I think this is good. I strongly recommend people to try this library extension, Chrome extension, to at least quickly verify if their sites are not using library versions that are vulnerable and you are putting your site users at risk.

Arturs Sosins: Actually, it would be great if they would put an online version where I can insert my web site and double-check it and tell me if there's something wrong with it.

Manuel Lemos: Well, if they can do that with the command line scanner, they can also put it in a site. That probably would be interesting.

Arturs Sosins: I would test the JS Classes.

Manuel Lemos: Yeah.

[Laughter]

Arturs Sosins: That's your site.

Manuel Lemos: Yes, you should. Although I do not use some of the most popular libraries that they know they are vulnerable, but OK, it's valid for any type of library. I don't know how it works in practice, if they just check version numbers or just check the actual code.

It would be interesting if they check the actual code because I don't know if you have watched last month. We had an hangout with a couple of people that are security-concerned, but more on the PHP world.

I mean, a OWASP developer, Jim Manico. He actually presented several things regarding good, security ractices. It is interesting that more and more people are being made aware about security concerns, and libraries like this one also are a good complement.

Generate Custom JavaScript Coding Guidelines with JSCode (19:01)

Manuel Lemos: Well, OK, we have covered this extensively. Let's move on to the next topic. The next topic is about something that I found unusual, at least for me, which also regarding code guidelines and stuff.

Let me share it here. OK, this one is a JS code which is basically a tool to generate and share your own JavaScript guidelines.

This is interesting because in some worlds, for instance, in PHP that never had strong system of guidelines, more recently, several developers, mostly related with I'd say frameworks and packages like that, they have gathered to define guidelines to use a single code structure, code formatting rules.

And this project here goes somehow the opposite direction, which is you can define custom guidelines which are not necessarily the same rules other people use.

So, how this works, you just click here on the Start button and you start defining some rules. I'm going to click here on some random options as to show what happens when you are done with all this possible choices.

And then, you continue and preview your JavaScript code guidelines. I can preview it. And it shows me the actual options that I picked and then it shows some code, which I trust this is some arbitrary sample code just to see how it gets formatted.

Let me increase the font here. And this is a bit unusual; I've never seen something like this. I was wondering if you could actually just validate some code with these rules, I can't find anything here.

Arturs Sosins: If they have an example code which can be formatted by the rules you specified, then most probably.

Manuel Lemos: They use some parser, right?

Arturs Sosins: Yes. So, maybe there could be some kind of standard that text editors could accept and format the code accordingly.

Manuel Lemos: This is the preview window, but if I click Save, what happens? It's all right, "The package was created successfully." And then...

Arturs Sosins: You can share.

Manuel Lemos: I can share, it just shares the URL.

Arturs Sosins: Yeah.

Manuel Lemos: So, it's not exactly something for an application, maybe.

Arturs Sosins: It's simply a web site link, it seems to me.

Manuel Lemos: It's just a web site link. It just links to a page of their own. And there is a list here of things that you should do with this library, create a package, choose the coding standard, you agree, share it with co-workers, question mark, and then profit. The problem here is question mark.

[Laughter]

Manuel Lemos: What you do, you do with these guidelines. Well, maybe it's something that you can probably in the future version of this site, they could actually generate some file format for editors as you suggested.

Maybe generate some library or some configuration for a library to parse the code and tell you if the code is OK or not, according to the rules that you defined.

Well, this is unusual because of that. I thought it would be worth mentioning because of that.

Arturs Sosins: I've actually heard about PHP standards because everyone is pushing their own standards, but I've never heard problems with JavaScript standards. Have you heard any arguments, discussions about it?

Manuel Lemos: Well, I don't know because I think from the beginning, the code of JavaScript tends to use those rules that are already being used by Java developers, like camel case for function calls. But as you may have seen in this site, they allow you  to define variations like tabs versus spaces, also the case, and many other parameters.

So, what is intriguing for me is what you do with that. Maybe in the next version of this library, they can actually add some things that are more useful to apply with those rules in practice.

Extensible JavaScript Code Auditing with ESLint (24:30)

Manuel Lemos: And talking about rules, moving on to the next topic which is somehow related, which is about ESLint. ESLint is yet another Lint project. We have talked a lot about JSHint and JSLint.

What's different about this one is that it is extensible. This means you can define new rules somehow. And this probably would be useful to use with the previous project, JSCode, which probably one can define a file with rules that would be enforced by this one, the ESLint.

And that probably will make it more useful than just having two projects that by themselves are not very useful because I do not see myself writing JavaScript code define new JavaScript rules. Although there is always people that would do that.

Well, that's my take on what this ESLint could be used for. I know it takes some configuration files and probably, it could take the rules from the JSCode that you define it with. That's basically just a user interface for defining rules.

I don't know. Did you try this project? Did you read this article about it?

Arturs Sosins: Yeah. When I first saw it, I was just like, "Oh, come on, another Lint."

Manuel Lemos: Yeah.

Arturs Sosins: Then when I read that... there's a feature that makes it different, is that it want you to define your rules. I was thinking, well no... You probably have used the Lint ones or Hint, is it really common that you would define your own rules?

Manuel Lemos: Right. That's...

Arturs Sosins: Did you have a need for it?

Manuel Lemos: Well, at least for JSLint, I don't have to define my own rules but I make exceptions to tolerate some demands of the rules that they imposed there by default.

So some rules, I want to make tolerable, so I just disable the warnings and all of a sudden, my code is clean. Because there are some rules there that are too pedantic, why would you want them there? I'm not remembering any rule at the moment, but I know that often I disable it.

Actually, you can even embed in JSLint and JSHint, I think. Both allow you to embed some commands that will define variations of the rules, the options of the parser, what it will complain or not.

And if you put that in your code, the next time you paste the code there in JSHint or JSLint, it will recognize that as initial options. And then, it will probably will not complain about things that you don't want to complain on.

Well, I started putting those command in my own code, because I really did not want to go through all those complains that I do not think are worth looking at. But OK, different people would have different rules, so it makes sense to have that defined exceptions.

But I'm not sure if it goes to the point of validating the format of the code. I do not remember but I think there are rules like, 'Ah, you are mixing tabs with spaces. So, you should either use tabs or spaces, not both.'

But other than that, it doesn't enforce that you use tabs for indenting or spaces for indenting or the number spaces and things that are defined in this JSCode formatting rules.

I think this JSCode thing's just more for code formatting than the actual code, if it is valid, if it has any problems. Well, that's what I got from this. I don't know if you have tried it and have a different perspective.

Refactoring JavaScript Code with GraspJS (28:56)

Manuel Lemos: OK, now moving on to another library which I also found a bit unusual which is Grasp.

It basically says that to "Search and replace your JavaScript code based on its structure rather than its text". At least for me, I read this like, Oh, this is a proper code refactoring tool, I think. I did not try it in practice but I could look at the examples.

And it says Skip to Demo, but to skip to demo just to scroll to this. When you're here, at least you have access to a sort of a command line and you need to actually...

Arturs Sosins: I think you can use the command from the example above and they should work, because they have a...

Manuel Lemos: What do you mean by command? Where is it?

Arturs Sosins: The first line of the first example. For example, grasp if test, operand equals end.

Manuel Lemos: Oh, I was not noticing it here.

Arturs Sosins: The first line is a command and then it outputs what it found in the file, in a JS file. And you can try the same commands and it just text.

Manuel Lemos: So, just copy and paste and it'll work.

Arturs Sosins: It should.

Manuel Lemos: Or maybe not. It doesn't. So I need to type this. It is not for lazy people like myself.

Arturs Sosins: Yes, so as you see from command, you can like queries structure of the code. If dot test, then you mean you are testing something and you can define an operator that would check for if statements which has end in them.

Manuel Lemos: Oh, I see. Well, basically, what it does is to actually show the result that is seen above.

Arturs Sosins: Yes, but probably you can redo replacements also. And I think maybe even something more. You can try typing help and see what commands it has.

Manuel Lemos: Well, help, it's...

Arturs Sosins: Yeah, nobody uses help, I know.

Manuel Lemos: No. It's probably Grasp...

Arturs Sosins: "grasp --help".

Manuel Lemos: And it shows the usage.

Arturs Sosins: Yeah.

Manuel Lemos: OK, the idea here is just to let people know that this is one of those things that is probably useful in a certain context on which you need to rewrite part of your code.

I don't know if you are going to use this much. I don't know. Have you ever needed to somehow refactor your code that probably could make use of a tool like this.

Arturs Sosins: Actually, I don't know. I had the need to refactor in Java, in C#, but I think I've never refactored anything much in JavaScript or Lua or some weakly-typed languages. Because you can't really efficiently do that in weakly-typed languages.

Manuel Lemos: Well, actually some time ago, probably not very long time ago, I have seen a PHP class that does that. It would parse the PHP code and then it would do replacements, like if you have variable names, such and such, replace it by something.

So it would not actually go through the text, the raw text of the code. It would actually parse it, so it make appropriate replacements. So it seems more like this Grasp, too. But Ok, it was for PHP. I did not know anything that could do this for JavaScript. So it's always good to know there are tools like this.

Arturs Sosins: And from the website, it seems its node only. You can't really inject it in the web page to try.

Manuel Lemos: Right. Now, everything command line-based in JavaScript seems to be based in Node. So, well, if you are not using Node, probably you are not a very good JavaScript developer.

[Laughter]

Manuel Lemos: Well, it depends. You can be a good JavaScript developer on the browser-side, but probably not be a great enthusiast of Node on the server-side.

Installing JavaScript packages with Composer PHP (33:46)

Manuel Lemos: Anyway, now moving on to the section which I'm going to comment something that was just released.

Arturs Sosins: Something that you did?

Manuel Lemos: Yes, but not JavaScript-specific. It's basically a feature of the PHP Classes and JS Classes sites, because they both share the same base code, which is to support that you can install packages using a tool named Composer.

This tool was originally thought for installing PHP packages, but when you have Web-based projects, you actually also have JavaScript, CSS and image files in your project. So there was a need to deploy them somehow.

And this Composer tool seems to have been inspired on Node Package Manager (NPM). So, it is a tool that takes some configuration for different aspects. And the configuration is actually defined in JSON format.

So, despite that it was mainly made for PHP, it takes a lot from the JavaScript world starting from the format of the configuration. But the reason why I'm talking about this is because PHP Classes and JS Classes site now supports Composer. This means that you can download and install packages from any of the sites using this Composer tool.

So for those that are not familiar, just to give a brief overview, basically this is a command line tool very much like those Node.js-based tools. Like we just mentioned, Grasp is also a command line tool for Node.js.

You install that, tool. Basically, you can download it in the PHAR format which is a sort of TAR with PHP code inside. It installs that PHAR archive somewhere. That PHAR archive contains classes and scripts for that tool. But that doesn't matter because you only deal with that one file and then after installing, you call it from the command line to install the packages you need.

And what you need to do is basically define a file in your working environment, maybe even production environment, that defines which packages you want from the repository so they get installed.

So here is a sample of a JSON file that needs to be called composer.json. You need to put it on a project directory and then run Composer and it will fetch and install the packages that you have defined here.

In this case, this example, there is package named "phpclasses/formsgeneration" and this Composer would install it if you just run this command her "php composer.phar install" and it then install the packages that you request. There are some configuration here to tell from where, what is the repository that you want to download and once it is installed, the package files are available on your directory.

For PHP, it even creates some autoloader scripts that will help you to actually load your packages without doing the require for every class script. You just do a single require statement here and then you just create an object over the class and the autoloader that is generated by Composer will include the necessary PHP scripts for actually loading the classes.

But this is for PHP. In a project that you need to also have JavaScript, CSS and image files, there is an additional challenge which is the fact that you have your project directory...

Actually I have a mistake here, it's not 'autoload/vendor'. It's 'vendor/autoload'. I need to fix this article.

Anyway, as I was saying, if you install packages, could be either packages from PHP Classes or JS Classes, it will install them inside the vendor/ directory. And the problem with that is that the vendor/ directory probably will not be under your probably... I mean, you should not put the vendor/ directory under your document root, for security reasons, or else the files that you have in there will be accessible via the Web.

So you should have probably a directory named Web and that would be your document root for your site domain. And that's where you would like to have your JavaScript or CSS or Images installed.

So the original Composer tool does not actually address that problem of making those files available under your documentary root. But fortunately, it can be extended with some plugins which are also some other packages that can be downloaded and installed and then it can do anything after installing the packages. So, it could for instance, copy the files to the Web directory.

And this is the part that makes it useful to install JavaScript packages. For instance, you could go on JS Classes and specify that you want to download some package with some JavaScript libraries, for instance, developed here by Arturs.

And it could tell with this plugins, tell Composer to move the relevant JavaScript files to your document roots so they can be available for your site pages. And the same goes for CSS or image files which also may need to go to some directory under the Web document root.

And this is basically the part that makes this useful for JavaScript projects, that may include probably a mix of service-side and browser-side of code. And the idea to seems to me very interesting because this makes it more systematic, the deployment of projects.

Arturs, have you ever tried the Composer before or any other tool like this for installing packages for your projects?

Arturs Sosins: I actually ran into Composer I know maybe about a year ago, maybe even more, when I needed one complex PHP software that was analytical artificial intelligent software for university purposes.

And that was the first time. Then I saw that it was using the Composer. I know I installed it on Windows which actually had a quite Windows installer and graphical interface, basically no command lines. And installation of the packages actually went smoothly.

And I was really impressed with it because this kind of stuff usually don't work so well and you need more configurations and something like that. But in this case, that was something great, really.

Yeah. But most probably, there are also other package managers. So, why did you choose this Composer exactly? Was it the most popular, the most mature?

Manuel Lemos: Well, first, I wanted to address the need of PHP developers. Several PHP developers have asked to have a Composer support so they can tell users, 'Go here and install it with Composer'.

Composer, by default, would fetch packages from Packagist, which is a different repository developed by the Composer developers. But the packages that are in PHP Classes and probably not in that other repository would need to have support of Composer in PHP Classes itself. So, that is what was developed. It has turned PHP Classes into also a Composer repository.

This means that if you want to download packages from PHP Classes and also JS Classes, you can just download them manually using either zip or tar.gz archives or you can use Composer to do this and follow all the process of combining packages probably from different sources, different repositories.

You can combine packages from PHP Classes with JS Classes, Packagist and probably others that exist. I don't know if there any other Composer repositories.

Yes?

Arturs Sosins: Basically, there are many packages that even in PHP Classes...

Manuel Lemos: Package Manager.

Arturs Sosins: No, I mean packages which depend on other classes in the same PHP Classes work site. So, basically if all developers would add their components to Composer, it would be just like downloading single package. And Composer will automatically install all the dependent PHP classes, right?

Manuel Lemos: Yes, that's the idea. Because one problem that Composer solves is also installing dependencies. For instance, if you have a package that requires several other packages, it will also install them. Because PHP Classes and JS Classes provides the author means to specify which are any other package that depends.

For instance, I have an OAuth package that depends on the HTTP package. So when you request to install OAuth package, it will install both. First one, then the other, because it's dependent.

Furthermore, the dependencies between these packages are already defined in PHP Classes and JS Classes because the site provide means to the authors to let them tell which package depend on what.

And this way, you can automate your installation and this makes it a good way to have it ready for users that probably do not want to think much about what depends on what. And if they use Composer, they have all the dependencies solved.

I remember I was talking about this OAuth package that I have in PHP Classes that depends on the HTTP package and very frequently, although the site leads on the page, 'Oh, this package needs this one. You need to also download and install it.', many users would not pay attention to that information and then complain in the forums, 'Oh, where do I get the HTTP class?'

Then I started changing the example's code and the first line says, 'Also download the HTTP class from this directory'.

But if they use Composer, they would not even have to notice that information because it would be installed automatically. They would just need to include that line that I mentioned above that, here, still has a mistake I need to fix it, it would require vendor/autoload.php and it define and the autoloader and every time you need a class, it would also include another class from the repository that is dependent.

And now, about your question, why this Composer and probably not other package installers? First, because there is this request from PHP developers to support it. And another aspect that I did not mention is that just like PHP Classes and JS Classes support Composer, it is extensible, and could add support to other packages like Node Package Manager and so on.

This is interesting and a few months ago, I talked about this with Michael Kimsal. I thought I was supporting a Node Package Manager, NPM. It is not yet supported but it is a possibility for the future. And then, Michael said, Oh but for JavaScript, it's also interesting to support Composer because many developers that use JavaScript are using PHP, because it's part of a whole Web project. So, here it is, it's all supported now. And I think it is useful already.

The only thing that is missing for now is that there is a need to have a plugin that will copy those JavaScript files to the document root wherever your application needs it.

There are several plugins developed by other people to do that, but I'll probably develop one myself and put it in PHP Classes. So if you want to install JS packages from JS Classes, you can specify that plugin and it gets installed and then it will do whatever is needed to copy the files to the right places.

I think this is useful. I don't know if you have thought about that. What do you think? Is this something that will be appealing to use?

Arturs Sosins: Yes, yes, definitely. As you said yourself, sometimes it's confusing with all the dependent PHP classes and other packages that you need to download. And it's also time-consuming. So, of course, that will ease the task of trying out some PHP packages because there's lots of dependencies.

Manuel Lemos: Right.

Arturs Sosins: Can you specify in PHP Classes though that are, for example, can you specify that class relate in some JavaScript class in JS Classes page? Can you specify the page dependency?

Manuel Lemos: Well, for now, I did not have support for that. For now, the only dependency that you can specify is for packages in the same site. But in your composer.json, you can define whatever it needs. You don't need to have them downloaded automatically. But when you define your project, you can define your composer.json.

Let me actually show something rather than just telling. So here, here, here, when you define here the packages that you need, here in the requires, you need "phpclasses/formsgeneration" but you probably also need "jsclasses/somepackage".

And then, you tell here below the repository to look up for package with that name. You can also have here a line for JS Classes or some other repository, whatever.

You don't have to just use packages from one repository. You can mix packages from different repositories. And this is what makes it more interesting because it addresses the needs of different developers.

It may be from PHP Classes or people that prefer some other repository and hopefully, everybody would be happy.

The only thing that for now is not available exactly in the official Composer distribution is having support for doing automatic login and private repositories. In the case of PHP Classes and JS Classes, they are not exactly private, but when you download certain packages, it requires authentication.

So there was a guy named Stephan Hochdoerfer, I'm not sure if I'm pronouncing his  name right, he's from this company, BitExpert, and he developed a patch and submitted a pull request to Composer so it can read user name and password from a separate file called auth.json.

And he submitted the pull request but I don't know why it was not accepted and merged yet. Probably the authors of Composer have different plans to implement a similar feature. I don't know, because he already asked several times why the feature does not get merged and he did not get a reply. There must be some reason.

It doesn't matter for now. What matters is that, if you use the official Composer it will prompt you the user name and password. So, with this patched version, it can read username and password from a file, auth.json that has this format.

You can specify for PHP Classes, I have this user name and password. For JS Classes, this is the name and password.

Well, since the official Composer version is not available and the PHP Classes distributed a patched version of composer.phar that is in this URL and in this link, which is basically the phpclasses.org/composerphar.

And you can install this version and it's the same version that you have on the official Composer, apart from this patch. It merge the patch by Stephan and it will work.

And that's it. That's the only difference that this patched version has, just to make it a bit less tedious to enter user name and password again.

But other than that, it's all the same. It's all useful. I've been using it extensively and it works very well. I even forget that there is some authentication going on.

Anyway, the idea here is just to encourage people to start using Composer more, because it is helpful to everybody to combine packages probably from different repositories.

And so, in PHP Classes and JS Classes, authors that have their packages in the Composer repository will earn additional reputation points that will increase their reputation for every package they have put in the repository.

For instance, I'm going to open here, let me see, it's a bit slow, OK, you can see that there are actions here that were recorded. I earned some additional points because I added all my packages to the Composer repository.

And the site makes it easy if you want to add all packages you have at once, in case you have many packages. So you don't add one at a time. I'm probably going to make this even easier to automatically add the packages to the Composer repository if there is at least one user that requests it. And probably the author doesn't even have to go through the screen to add the package to repository.

And that's basically it. Hopefully, this will encourage more people to use Composer. Let's see how this goes for the future of the practices of the authors.

Arturs, do you think they'll start using Composer more and enable your packages with Composer support? What do you think?

Arturs Sosins: You're giving the extra points, I'm doing it already.

[Laughter]

Manuel Lemos: Oh, silly me. Why did I ask?

[Laughter]

Manuel Lemos: OK, hopefully, that would encourage more and more people to do it. But as I said, I'll figure out something to do it more automatically.

The author doesn't have to even go there, as long as there is somebody requesting it. And that somehow will reflect on the perceived qualification of the developer that is using this Composer because he's conscientious to make it easier for their users to install their packages.

JavaScript Innovation Award Winners of October 2013 (57:51)

Manuel Lemos: Basically, that's it. We're reaching the end of our podcast. And we're finally going to one last session on which we comment on the latest winners of the Innovation Award. Unfortunately, the month that we are commenting now which is, I think... Well, we are in December but it was the packages that were announced with the winners of...

Arturs Sosins: October?

Manuel Lemos: Is that October? Exactly, October. So, unfortunately, there is only one package to comment because for some reason, that was not a very good month.

Arturs Sosins: Well, this means September was a not very good month, right?

Manuel Lemos: No. It was the ones that were released in October, basically only one package.

Let me share the screen here, because for some reason it stopped sharing the screen.

In this case, there is only package by MarPlo from Romania, I think, right?

Arturs Sosins: Yeah.

Manuel Lemos: So, basically this package what it does is to convert XML documents to the JSON format. I don't know if these days if people are using many XML documents but I think this makes sense in probably some context that you need to deal with XML. Because, at least in my opinion, since JSON got popular, nowadays people are not using anything else to store structure data in files.

What do you think?

Arturs Sosins: Yeah, that's the funny thing. You could say that JavaScript was built for XML document because they were using a DOM tree in the HTML and HTML is a version of XML.

So kind of was build for that purpose but on the same page, using JSON is even more easier because it checks away... basically JavaScript objects. So I'd say that this is actually a quite a useful component that could be used on probably any API that uses XML. And automatically, you can load it to JSON, thus quite easy, useable for JavaScript.

I have an example, I have screen shared the example of this package. And basically, you could see how it parses the document. You could see that... Do you see the screen share?

Manuel Lemos: Yeah.

Arturs Sosins: So, you could see that it parses the attributes separately and for some reason for example, we have a title and then the text inside. And then, it's parsed as a title key  and then it has another key attached.

I assume it's something similar to HTML Nodes, nodes that probably has a text node inside a specific tag or something like that. That is why it's parsed this way.

But this library is specific, I'd say and well, but still it seems quite useful. And basically, as I said, you can turn any XML-based API into JSON API through this.

Manuel Lemos: Right. Because I think the fact that, as you said, in the browser, you already have a DOM library that you can use to parse your XML or even HTML. And it's very easy to surface the objects and then extract it in JSON format so it's all simplified. But at least, I did not know a library that could do this. So kudos to MarPlo for this library.

Well... Yeah?

Arturs Sosins: Actually, when thinking about, I don't know, even in PHP or other languages, as I am lately using Lua, in every one, JSON is more easier to use than XML.

Manuel Lemos: Yes, That because typing XML is tedious. Because if you have a long tag to open, you also to close the tag. And it's mandatory for XML.

So for people to write JSON is more convenient because you don't have to type the whole name of the tag of the attribute again to just close the tag. And probably that's the biggest complain about the XML and that's why if you talk about XML these days, just think 'Oh, that's so 2000'.

Arturs Sosins: Exactly.

Manuel Lemos: You are old. You are obsolete.

Arturs Sosins: So, soon there will be a case when we won't be working this DOM tree but rather DOM JSON representation or something like that.

Manuel Lemos: Well, practically, it could all be that, instead. I think you have to go through the DOM libraries to get the children and stuff. But it could be an idea for the future. But for now, we need to deal with the past and that's the way it goes.

Arturs Sosins: At least, we could serialize HTML into JSON and then deserialize it back to the HTML tree.

Manuel Lemos: Yeah. At least for server-side, most APIs these days, they no longer use XML, even less SOAP or...

Arturs Sosins: True.

Manuel Lemos: XML RPC and the other APIs with XML formats, it's all JSON now all over.

Well, good to have this library. Unfortunately, we do not have any other to comment this month. We'll have more next month, there were more nominees. But we need to wait until then.

Conclusion (1:04:25)

Manuel Lemos: So, with this, we practically end this hangout. I would like to thank you again, Arturs, for coming, for filling in again for Michael Kimsal that is still very busy.

Arturs Sosins: For busy Michael?

Manuel Lemos: Yeah, well, he got so popular, now everybody demands his work.

[Laughter]

Arturs Sosins: ...we'll start to pay him for showing up on podcast.

Manuel Lemos: Yeah, that's probably a hint. He doesn't show because I do not pay him well. Well, anyway, let's see if he can make it next time.

With this, we have ended this podcast. On my behalf, that is all for now. Bye.

Arturs Sosins: Bye.

[Music]