Contents


The busy JavaScript developer's guide to LoopBack, Part 2

Models

Define and use models and data sources

Comments

Content series:

This content is part # of 6 in the series: The busy JavaScript developer's guide to LoopBack, Part 2

Stay tuned for additional content in this series.

This content is part of the series:The busy JavaScript developer's guide to LoopBack, Part 2

Stay tuned for additional content in this series.

Welcome back! In Part 1 of this series, you took a quick look at the LoopBack server-side JavaScript framework, installed it, scaffolded a basic application, and explored some of LoopBack's API tools. In this second installment, you'll see how LoopBack handles models— objects that represent how to store and retrieve data. LoopBack defines model objects via several mechanisms, with different persistence mechanics.

Before we get to that, we need to decide what we're going to model and store.

The LoopBack model

To demonstrate how to apply models, we'll develop a simple API — but first, we need to determine what data the models will use, and how. Every sample like this requires a relatively simplistic problem to solve — something that affords a decent amount of complexity, so that we can see how to model more complicated definitions and relationships, but not so elaborate that we spend most of our time explaining the (fictitious) application model. Toward that end, I thought it apropos of the current millennium to build out a system (exposed as an HTTP-based web API) that will track all of the various characters in superhero movies. After all, how can you truly understand the intricacies of the modern superhero movie if you don't know who these superhero characters are, what they can do, and what they're vulnerable to? (Just go with me on this — remember, it's a demo.)

Our Superhero-Information-as-a-Service (SIaaS) application will start by tracking some basic character information: the character's real name (first and last); their "code name" (their superhero name), if they have one; and their place of origin — terrestrial (Earth-based) or otherwise. Lastly, we'll also track their "karma" index — an abstract number that indicates their relative "goodness" or "badness" by using simple upvote/downvote data, where an upvote will reflect a "good-guy act," and downvotes will register their "bad-guy acts." This way, we can track certain characters who don't fit into simple good or bad buckets.

In time, we'll also note those characters who are part of superhero teams, which could be tricky given that many of them are members of multiple alliances — but we'll defer that to a later piece in the series. For now, we'll focus on storing "simple data" (the strings for names, places of origin, and so on) and building a mechanism to allow upvotes (heroic acts) and downvotes (villainous acts) to indicate the character's cosmic karma, but without allowing users to directly modify that karma (that is, we want to encapsulate some data so that it can be manipulated only through well-defined points of entry).

If you can imagine a method, LoopBack probably has it, as long as it's not model-specific.

Defining the model

We'll begin by defining a simple model class, using the LoopBack CLI to scaffold out the files and core. If you've not done so already, install the lb command-line tool by using npm to install the loopback-cli package. Where we used the hello-world example in Part 1, we'll instead run lb to generate an empty-server application, which will provide a blank slate.

    lb

         _-----_     
        |       |    ????????????????????????????
        |--(o)--|    ?  Let's create a LoopBack ?
       `---------   ?       application!       ?
        ( _U`_ )    ????????????????????????????
        /___A___\   /
         |  ~  |     
       __'.___.'__   
        `  |  Y ` 

    ? What's the name of your application? heroes
    ? Which version of LoopBack would you like to use? 3.x (current)
    ? What kind of application do you have in mind? empty-server (An empty LoopBack 
    API, without any configured models or datasources)
    Generating .yo-rc.json


    I'm all done. Running npm install for you to install the required dependencies. If this fails, try running the command yourself.


       create .editorconfig
       create .eslintignore
       create .eslintrc
       create server/boot/root.js
       create server/middleware.development.json
       create server/middleware.json
       create server/server.js
       create .gitignore
       create client/README.md

This command spins up an empty application with the core basics we need. Note that the client directory contains only a README file; this is a reminder that LoopBack generates only the server side/API side of a full-stack application and makes no assumptions as to how the API can or should be consumed. LoopBack provides the models and controllers; views are yours to build.

To verify that the application has been generated, run node in the application code directory and browse to http://localhost:3000 to view the JSON response containing the current uptime of the server (as seen in Part 1). Now we need to build out a model. Our initial model type will be the Hero type, representing a character from the superhero universe. But first, we need a place to which these models will be stored and from which the data will be retrieved. In LoopBack terminology, this is a data source. An empty-server application by default has no data sources defined. Because this is a green-field application, with nothing preexisting, it's easiest for now to define an in-memory data source using the lb datasource command:

    Teds-MBP15-2:code ted$ lb datasource
    ? Enter the data-source name: memdb
    ? Select the connector for memdb: In-memory db (supported by StrongLoop)
    Connector-specific configuration:
    ? window.localStorage key to use for persistence (browser only): memdb
    ? Full path to file for persistence (server only): ./mem.db

You'll need to define a unique name for your data source; because LoopBack can work with multiple data sources in a single application, each must have a distinct name. After naming the data source, you'll answer a series of configuration questions; in the case of the in-memory database, only one configuration element is required — the name of the file to which to store the data.

Under the hood, the lb tool is simply laying down some files. Specifically, it's configuring data sources in the file server/datasources.json, which is a simple JSON file:

    {
      "memdb": {
        "name": "memdb",
        "localStorage": "memdb",
        "file": "./mem.db",
        "connector": "memory"
      }
    }

The values in each JSON element will vary depending on the data source type, but they'll typically correspond one-to-one with the configuration parameters that the lb tool asked for at the command line. In this case, if I suddenly decide that memb.db is a terrible name for a database storage file, and prefer database.json, it's trivial to make the change: just change the "file" entry in the previous JSON listing, and if the data already exists, rename the file to match the new name. Bam! Data storage refactored. Switching from localStorage to a different type of data source (such as a MongoDB instance or relational database) would require more work, but we'll postpone that for now.

After the data source is configured, we can add models. In a green-field application such as this one, the lb tool makes it easy to concoct a simple model straight from the command line, using lb model; however, as soon as you do, you'll be greeted with an interesting choice:

    $ lb model
    ? Enter the model name: Hero
    ? Select the data-source to attach Hero to: memdb (memory)
    ? Select model's base class (Use arrow keys)
      Model 
    ? PersistedModel 
      ACL 
      AccessToken 
      Application 
      Change 
      Checkpoint 
    (Move up and down to reveal more choices)

The first two queries are self-explanatory: the name by which the model should be known and the data source to which the model is attached. But the model's base class deserves explanation.

A Model (that is, a model that extends the LoopBack Model type) is different from a PersistedModel in that the PersistedModel introduces the basic methods we expect for interacting with a database: create, update, delete, find, findById, and so on. Not all objects in a system need to or should be stored in a database, however, so LoopBack offers us the opportunity to extend the base Model type as a way of keeping model types simple when they don't need persistence. The Model type offers a number of useful event methods (such as changed and deleted) that are interesting in their own right, including a checkAccess method that can be used to provide discretionary access to particular objects.

PersistedModel offers these methods:

  • create: Create an instance and save it to the database
  • count: Return a count of objects that satisfy the passed predicate criteria (if any)
  • destroyById: Remove an instance from the database
  • destroyAll: Remove the whole collection of these model objects
  • find: Find all model instances that meet the passed criteria (if any)
  • findById: Find a given model instance by its unique identifier
  • findOne: Find the first model instance that meets the passed criteria (if any)
  • findOrCreate: Find one record that matches the filter object passed in; if no such object exists in the database, create the object and return it
  • upsert: Update or insert (borrowed from MongoDB terminology), choosing one or the other depending on whether the object already exists in the data store
  • updateAll: Modify all instances that meet the passed criteria with the new data (similar to the classic relational UPDATE statement)

In addition to the above static methods, PersistedModel ensures that any instance will also have similar convenience methods for single-object use:

  • destroy: Remove this object instance
  • getId/setId: Return or modify the unique identifier for this object
  • isNewRecord: Return whether the instance is new
  • reload: Reload this instance from the database, discarding any changes that have been made to the object in the interim
  • save: Store the object to the database

This is hardly an exhaustive list; see the LoopBack API documentation for the complete list of methods and details on each. If you can imagine a method, LoopBack probably has it, as long as it's not model-specific. All of these methods, owing to their I/O nature and the corresponding conventions in NodeJS, take callback functions to invoke when the operation is complete.

Our Hero class will be extending the PersistedModel type. Selecting PersistedModel brings up more questions:

    $ lb model
    ? Enter the model name: Hero
    ? Select the data-source to attach Hero to: memdb (memory)
    ? Select model's base class PersistedModel
    ? Expose Hero via the REST API? Yes
    ? Custom plural form (used to build REST URL): Heroes
    ? Common model or server only? server
    Let's add some Hero properties now.

    Enter an empty property name when done.
    ? Property name: Codename
       invoke   loopback:property
    ? Property type: string
    ? Required? Yes
    ? Default value[leave blank for none]: 

    Let's add another Hero property.
    Enter an empty property name when done.
    ? Property name:

Many web API developers want to expose the defined models via a REST-like API endpoint; LoopBack takes that into account, so selecting Yes to the "Expose via REST API" question will automatically define some endpoints and URL parameters as a convenience. Because the plural form of the model's name isn't always machine-inferable, LoopBack asks us for the plural form, so that the endpoint can be grammatically correct.

Then, because the model definition is often shared by both front end and back end, LoopBack asks if the model's definition should be exported by being shared in a common directory between client and server. Since the application is server-side only, choosing server keeps the code self-contained.

Lastly, LoopBack will ask you to define the properties of the model type. The first property is the hero's code name, which is a string, must be present, and has no default. LoopBack will continue to ask for properties to define until you enter a blank name for a property.

Model definition and file structure

When you're finished, the result will be a pair of files defined in the server/models directory, hero.js and hero.json. The JSON file will be a definition of the Hero type that reflects the choices made at the command line:

    {
      "name": "Hero",
      "plural": "Heroes",
      "base": "PersistedModel",
      "idInjection": true,
      "options": {
        "validateUpsert": true
      },
      "properties": {
        "Codename": {
          "type": "string",
          "required": true
        },
        "FirstName": {
          "type": "string"
        },
        "LastName": {
          "type": "string"
        },
        "Origin": {
          "type": "string"
        },
        "Karma": {
          "type": "number",
          "required": true,
          "default": 0
        }
      },
      "validations": [],
      "relations": {},
      "acls": [],
      "methods": {}
    }

Defining additional methods

The hero.js file is an opportunity for you to add additional methods to the Hero type by taking the passed parameter (the Hero prototype object) and adding the desired methods, such as upvote and downvote methods to adjust the karma of the hero appropriately. (We'll see those methods later; for now, we're merely fleshing out the data fields/properties.)

The full set of options available in this model definition file is described in LoopBack's documentation, but some features are visible in the previous listing:

  • base: The base type of this model. We've discussed Model and PersistedModel, but LoopBack models can also extend some of the predefined model types, such as User, Email, and others.
  • idInjection: This indicates whether LoopBack will be responsible for managing the id field in the model, which is assumed to be the primary key for that model. This will default to true for most models.
  • relations: Defines relationships that this model has to other models; we'll discuss relationships more in Part 3 of this series.
  • validations: This portion of the model allows us to define validations on properties. LoopBack's current documentation states clearly that this option is not yet implemented. Presumably, in a future version, we'll be able to set minimum length, maximum length, and regular-expression-based validations that LoopBack will automatically enforce. For now, any validations require handwritten code.
  • acls: LoopBack, unlike most of its cousins in the NodeJS world, has a rich and powerful access control model, allowing you to define roles, permissions, and access control lists to manage users' access to objects. We'll explore LoopBack's access control in a future article.

The model definition file provides the API's basic structure, but LoopBack's developers recognize that not all attributes can be captured in data format like JSON; sometimes you have to write code. Thus, the hero.js file allows the developer to take the passed-in Hero object and add whatever attributes seem appropriate to the model prototype object.

For example, we mentioned that the "goodness" of a hero is measured by their karma: Every time they do a good act, their karma should go up, and every time they do a bad act, it should go down. Normally, influencing a hero's attribute would involve doing a PUT request, passing in new data as part of the request's body. This isn't quite the same; we'd prefer to have karma influenced only by upvotes and downvotes. In LoopBack, we can accomplish that by creating two methods on the Hero, and then registering them as remote methods, meaning they get incorporated as part of the overall Hero web API.

You'll add these methods in the (empty) hero.js file. First, define the two methods on Hero, and then register each as a remote method:

    module.exports = function(Hero) {
        Hero.prototype.upvote = function(cb) {
            var response = "Yay! Hero did a good thing";
            this.Karma += 1;
            this.save(function(err, hero) {
                if (err) throw err;

                cb(null, hero);
            });
        };
        Hero.remoteMethod('prototype.upvote', {
            http: { path: '/upvote', verb: 'post' },
            returns: { arg: 'result', type: 'object' }
        });

        Hero.prototype.downvote = function(cb) {
            var response = "Boo! Hero did a bad thing";
            this.Karma -= 1;
            this.save(function(err, hero) {
                if (err) throw err;

                cb(null, hero);
            });
        };
        Hero.remoteMethod('prototype.downvote', {
            http: { path: '/downvote', verb: 'post' },
            returns: { arg: 'result', type: 'object' }
        });
    };

Note that the methods are defined on the prototype of Hero; this is necessary to define them as instance methods (that is, on each instance of Hero). Without that definition, LoopBack would assume them to be static methods, associated with no particular instance of Hero (and therefore would be unable to reference which Hero's karma is being up- or downvoted). Once the Karma has been appropriately modified, we rely on the built-in save method that Hero inherits from PersistedModel to write the Hero's new data back to the data source.

Registering remote methods is relatively straightforward, involving a single method call, remoteMethod(), which takes the name of the method to register and some data about the URL at which to expose the method: the HTTP path and verb, some description of the arguments (in this case, none), and the return value.

This might seem excessive, but the net result is that LoopBack now has direct information about the endpoint you wish to expose, and with that, it can add these two endpoints into the Swagger UI exposed in the Explorer view of LoopBack's GUI.

Model validation

We can also add validation code to verify that no two heroes select the same code name by adding in a validation constraint to hero.js:

    module.exports = function(Hero) {

        // ... as above

        Hero.validatesUniquenessOf('Codename');
    };

This makes use of the validatesUniquenessOf method that PersistedModel gets through the Validatable mixin, which includes the following methods:

  • validatesAbsenceOf: Validates absence of one or more specified properties; that is, to be considered valid, a model should not include a certain property, and will fail when the validated field is not blank.
  • validatesExclusionOf: Validates that the property's value is not in a range of values.
  • validatesFormatOf: Requires a property value to match a regular expression.
  • validatesInclusionOf: Requires a property value to be within a range of values. (This is the logical opposite of validatesExclusionOf.)
  • validatesLengthOf: Requires a property value's length to be within a specified range.
  • validatesNumericalityOf: Validates that a property is numeric.
  • validatesPresenceOf: Requires that a model has a value for the given property. (This is the logical opposite of ValidateAbsenceOf.)
  • validatesUniquenessOf: Validates that a property value is unique for all models. Note that not all data stores support this; as of this writing, only the in-memory, Oracle, and MongoDB data source connectors support this.
  • validate: Allows for a custom validation function to be attached.

Additionally, Validatable adds an isValid() method to each instance, so you can verify that an object is valid at any time (before sending it back across the wire as a result, for example) without explicitly trying to store the object.

Just to be clear, Validatable and its methods are accessible from any model inheriting from the basic Model type — an object doesn't need to be a PersistedModel to benefit from validation.

Discovering models

For developers with existing databases that cannot be rebuilt from scratch, LoopBack hasn't forgotten you. LoopBack offers the ability to discover models from existing data sources, such as relational database tables and schemas (MySQL, PostgreSQL, Oracle, and SQL Server are currently supported). This is a once-run process that generates the model definition files, and then LoopBack uses those as it would any other model definition.

The LoopBack documentation suggests that this would be done in a standalone NodeJS script, such as:

    var loopback = require('loopback');
    var ds = loopback.createDataSource('oracle', {
      "host": "oracle-demo.strongloop.com",
      "port": 1521,
      "database": "XE",
      "username": "demo",
      "password": "L00pBack"
    });

    // Discover and build models from INVENTORY table
    ds.discoverAndBuildModels('INVENTORY', {visited: {}, associations: true},
    function (err, models) {
      // Now we have a list of models keyed by the model name
      // Find the first record from the inventory
      models.Inventory.findOne({}, function (err, inv) {
        if(err) {
          console.error(err);
          return;
        }
        console.log("\nInventory: ", inv);
        // Navigate to the product model
        // Assumes inventory table has a foreign key relationship to product table
        inv.product(function (err, prod) {
          console.log("\nProduct: ", prod);
          console.log("\n ------------- ");
        });
      });
    });

To be available to use at runtime, the output must be written to a file (usually common/models/model-name.json), and then manually registered in the server/model-config.json file.

For data sources that aren't relational databases, LoopBack can also infer a model from an unstructured (JSON) data instance, meaning a MongoDB database, a REST data source, or a SOAP data source. You'd do this in a manner similar to a relational database: Obtain an exemplar instance from which to do the inference, and pass it to the data source's buildModelFromInstance() method. The return is the generated model type, usable in the same fashion as our Hero object from earlier. LoopBack demonstrates an example of this using a raw JSON object:

    module.exports = function(app) {
      var db = app.dataSources.db;

      // Instance JSON document
      var user = {
        name: 'Joe',
        age: 30,
        birthday: new Date(),
        vip: true,
        address: {
          street: '1 Main St',
          city: 'San Jose',
          state: 'CA',
          zipcode: '95131',
          country: 'US'
        },
        friends: ['John', 'Mary'],
        emails: [
          {label: 'work', id: 'x@sample.com'},
          {label: 'home', id: 'x@home.com'}
        ],
        tags: []
      };

      // Create a model from the user instance
      var User = db.buildModelFromInstance('User', user, {idInjection: true});

      // Use the model for create, retrieve, update, and delete
      var obj = new User(user);

      console.log(obj.toObject());

      User.create(user, function (err, u1) {
        console.log('Created: ', u1.toObject());
        User.findById(u1.id, function (err, u2) {
          console.log('Found: ', u2.toObject());
        });
      });
    };

For the unstructured data exemplar, it might be easier to run this as a bootstrap script that executes as part of the server's normal startup sequence. But unless the structure of the data is constantly changing (perhaps even without the developers' knowledge), it makes more sense to transform this into a model definition file one time rather than use resources to reparse the data instance every time the server starts.

Model API and the LoopBack GUI

If you start the server again (run node in the root directory) and browse to http://localhost:3000/explorer, you'll see the Hero type has already been wired up. You can use the interactive GUI to create new heroes — for example, using the POST /Heroes endpoint to add one to the system, and then using the GET /Heroes to see the full list of all that have been added.

Although it doesn't replace unit tests that exercise the API, the LoopBack GUI makes it easier to test the API interactively during development.

Model bootstrapping

Any system usually has a set of objects that should be bootstrapped into existence. A few heroes are consistently part of any superhero universe, so for convenience, we'd like those preloaded into the system anytime it starts. LoopBack provides a hook for this type of bootstrapping code — any code found in the boot subdirectory will be executed when the server starts, so all you need to add there is a file that exports a function to be invoked during startup. This function receives a single parameter — the application object — which we can use to obtain the model prototype object and create new objects if they don't already exist:

    module.exports = function(app) {
        app.models.Hero.count(function(err, count) {
            if (err) throw err;

            console.log("Found",count,"Heroes");

            if (count < 1) {
                app.models.Hero.create([{
                    Codename: "Superman",
                    FirstName: "Clark",
                    LastName: "Kent",
                    Karma: 100
                }], function(err, heroes) {
                    if (err) throw err;

                    console.log('Models created:', heroes);
                });
            }
        });
    };

Any model type defined will be a member of the app.models object, so throughout the codebase, anywhere you have reference to the application object, any of the models are a simple property dereference away.

Conclusion

You'll discover more details about LoopBack models, both as we explore them in this series and as you delve into LoopBack on your own, but most of what you'll find will be variations on the themes we've covered here; what you've seen so far is enough to get you started working with LoopBack models. However, a larger topic awaits us in Part 3: how we construct the relationship between models. LoopBack offers some structural support for relationships, but modeling relationships — particularly across different kinds of data storage systems — is not always straightforward and intuitive. Although LoopBack tries to abstract away the underlying data, it won't be able to handle all scenarios in all situations.

For now, though, it's time to move on. Happy LoopBacking!


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Web development, Open source
ArticleID=1045053
ArticleTitle=The busy JavaScript developer's guide to LoopBack, Part 2: Models
publish-date=04242017