CFML ORM — NOT NULL Constraints, Cascade, and Inverse

I’m deciding to make mini-blogs about the things I find out using CFML ORM. Hopefully these entries will save you time and sweat later.

I think it’s generally a good practice to leave properties as NOT NULL by default. I tend to see fewer bugs like orphan records that way. But I’ve ran in to an issue over and over again when dealing with relationships and the cascade attribute.

Maybe you have this same use case — a one-to-many relationship to which you want to cascade changes. Maybe an User and Comment? (A User component is usually my culprit here.)

All seems pretty straight forward, right? But when I try to delete a user, I get an error like the following:

[Macromedia][SQLServer JDBC Driver][SQLServer]Cannot insert the value NULL into column 'FK_user', table 'my_app.dbo.comments'; column does not allow nulls. UPDATE fails.

This has confused me like crazy. I don’t want NULL in the column. I want to delete it. Usually my fix has been to remove notnull="true" from the comments side of the relationship. (And I’m not the only one.) This works, but has all the drawbacks of having a nullable field when we don’t want to.

So what’s the fix?

inverse="true"

Yup. That’s it. I found the answer posted in the comment thread of an older article by Nicholas Tunney.

Make sure to add it to entity that owns the foreign key (User in our case).

There’s a good write-up here on inverse in Hibernate that helped clear some things up for me. The gist of it is that setting inverse="true" takes the SQL down from two inserts and an update to just two inserts.

Anyway, I’ve been bit enough by this behavior to document it here. Hope this helps someone else.

CFML ORM — NOT NULL Constraints, Cascade, and Inverse

ColdBox Configuration Gotchas

Overview

We recently had some memory issues we were investigating that were only happening on our dev and local servers. It led to a few interesting discoveries that I thought I would highlight here.

There are many “development only” settings in the config/ColdBox.cfc configuration file. In our naïvety, we turned them all on for our local and dev environments. We were trying to avoid having to remember to fwreinit our application as that was something we didn’t deal with in our homegrown system.

Well, even with turning on all those settings, I think we still basically reinited our application the same amount, not sure if we had missed a “development only” setting. So, it didn’t save us any time. It actually cost us more time and, as I will show, introduced a situation where our memory usage got quickly out of hand.

Let’s go over two settings that I think are a bit of anti-patterns (and the docs agree with me, though I ignored that when I started 😉):

wirebox.singletonReload = true

We’ll start with the more innocuous one. This setting promises to reload all singletons on every request. At first glance, this sounds great. No more remembering to reinit your application when you change a singleton. It also, however, introduces a few drawbacks.

First, it can be a lot of overhead. Singletons are a great pattern because they don’t need to be recreated over and over again. This setting throws all those savings away and the overhead added to recreate each one can be quite substantial, even up to a few seconds per request.

Second, it’s not thread safe. Every request, including ajax requests, will recreate all singletons. If a subsequent request tearsdown and rebuilds all singletons while the first one is in process, the first one could have some funky errors. I say could because it is a classic race condition, the worst things to debug. You might have run in to this yourself and never known why the error only sometimes happened. Here’s a likely candidate.

So what’s the solution? fwreinit when you need to reload a singleton. Yes, when you are working on a singleton exclusively, it will be reloading every request, but when you are not you are gaining a nice performance boost and avoiding some nasty race conditions.

modules.autoReload = true

On the surface, this setting looks like it has all the same benefits and drawbacks as wirebox.singletonReload = true. It can add a lot of overhead and is not thread safe. The major issues come from some interactions with common modules.

The module that caused us a bunch of issues was cbjavaloader. This is not due to any fault of cbjavaloader, but rather our understanding of it (or lack of, in this case). Every time a module that used cbjavaloader was reloaded, all the jars would be loaded in to memory. Combine that with reload every module on every request and you can see how memory usage could explode!

(On a side note: I think there’s a bug with cbjavaloader not remembering previously loaded urls. I’ve filed a bug.)

Recap

Now, to Ortus’s defense, they have discalimers on both of those settings about how it could make your application unstable. But I want to circle back to what I said at the beginning about how our team still always includes fwreinit when we are developing because we have no idea if we got all the settings. In our case (and I would venture to guess many others) we not only are not getting any benefit from these settings, but they are actively causing us more issues.

To that end, I’ve filed a request to remove modules.autoReload from the next major version of ColdBox, ColdBox 5. If you agree or disagree, come add your voice to the ticket.

Oh, and probably go turn those settings off in your own app.

ColdBox Configuration Gotchas

Snapshot Testing in TestBox

Snapshot testing is the latest craze. Popularized by Facebook’s Jest testing framework, snapshot testing is a simple way to add regression testing into your application.

What is regression testing, you ask? Regression testing is testing that the output did not change from what it previously returned.

Take the following example:

In this handler we are retrieving a list of users, optionally filtered by the request context. We can use ColdBox’s integration testing capabilities to test that this code is coming back correctly.

This is a great test! It will make sure that this users endpoint returns the values we expect.

Fast-forward a month or two (or maybe just a few weeks) and there is a new property on the User bean. With our current approach, our users endpoint test will fail, even though the logic hasn’t changed. We’ll need to go update our tests to match.

Not with snapshot testing. Let’s take a look at the same test using snapshot testing:

When running this test for the first time, the tests will fail mentioning that no snapshots exist.

Creating and updating snapshots is a deliberate action with snapshot testing. You have to pass a updateSnapshots url flag to create or update the snapshots for the tests ran.

Matchers that have their snapshots updated always return true. In addition, a debug message is logged with the name of the snapshot created.

You can see the snapshots created in your tests/resources folder:

Now, running the tests without the updateSnapshots url flag returns green.

The real power of snapshot tests come when you make changes. Let’s add an email field to the User bean and return it. When we run our tests now, we see the following failure:

This failure gives us the opportunity to check what changed in our system and if we were did break some existing functionality. If we did, our tests helped us catch it! If not, we can now update these snapshots by running the tests with the updateSnapshots url flag.

A really compelling use case for this is testing HTML. In fact, the snapshot intelligently normalizes HTML output so that simple indentation changes should cause the snapshot to fail.

To get started with snapshot testing in TestBox, install the testbox-snapshots module from ForgeBox. Then add the matcher in your beforeAll method and you’re good to go!

For more precise changes, make sure to include the optional java library in your load paths. It produces a diff of the files instead of just outputting the contents in their entirety. You can enable this by including the following in your tests/Application.cfc:

this.javaSettings = {
    loadPaths = [ "testbox-snapshots/lib" ],
    reloadOnChange = false
};

And that’s it! Snapshot testing is another tool in your testing tool belt. You can get started with box install testbox-snapshots --saveDev in your project right now.

One great use case that I’ve seen is using snapshot testing to start to get a hold on a legacy app. Add a snapshot assertion, create the snapshot, then start refactoring with confidence!

Snapshot Testing in TestBox

dev.Objective() Wrap-up

Well, dev.Objective is wrapping up now. This week has been my first week presenting at a conference, and I’m grateful to everyone that came out to hear what I had to say. Links to the slides are available below.

CFML Sessions for Dummies
Live Testing a Legacy App

Overall, I enjoyed the conference. As a CFML developer, I gravitated more to the CFML specific tracks and even wished there was more of it. After 3 days (4 with Into the Box) I am more than ready to head back home. I think I would just enjoy a 2-day conference track just as much, maybe even more.

One thing I especially liked was that I can’t remember hearing any one apologize for CFML here at dev.Objective(). That was a welcome change from CFSummit, where I feel that’s the pitch of the entire conference.

My favorite sessions were definitely the ones on Lucee. It was really neat to see behind the curtains, if you will, of Lucee — seeing the build happen, seeing a bug get fixed, seeing the community in action. I even entered a feature request (LDEV-888) yesterday that was added within 12 hours! Lucee is a breath of fresh air coming from ACF. I hope that I can get better at contributing back. I now know that test cases are just TestBox cfcs (like this), so I can definitely include one with a bug report.

I think my biggest complaint about dev.Objective() would be the cost. Both the conference and the hotel were pretty expensive and made it so my entire team couldn’t come. Hopefully those items can be addressed in the future in some way.

These conferences always remind me of how good the active CFML community is. I hope we can all work at getting more of the community active and contributing!

Until the next CFML conference.

Cheers,
Eric

dev.Objective() Wrap-up

Into the Box Wrap-up

Into the Box 2016 just finished up. What a great conference! It was my first time speaking and I feel like it was the perfect venue — only 30 minutes long in front of a small, supportive group about a fairly specific subject matter. Thank you to everyone who came to my sessions and supported me!

Below are the links to my slides for the presentations:

Mixing Up the Front-end with ColdBox Elixir
Integration Testing in a Modern World

I’m really excited for Into the Box next year, as well. The plans are right now to expand Into the Box to a 2-day conference all focused on Box products. I think that is a good idea. I really enjoyed how specific the content was, and I definitely have more to learn about this awesome framework and the tools around it. Looking forward to it!

Next up is dev.Objective(). I have two presentations there as well. If you’re in town, I hope to see you there.

CFML Sessions for Dummies
Live Testing a Legacy App

Into the Box Wrap-up

Multi-Engine CI in Travis with CommandBox 3.1.0

CommandBox 3.1.0 is just around the corner and comes with a host of amazing features! While the new ForgeBox integration deserves an entire post on its own, I want to talk about starting different CFML engines all from CommandBox.

Brad Wood outlines it on his blog here. Basically, the CFML engines are now packages on ForgeBox, and you can start a specific engine by setting the cfengine parameter for the server start command. It has never been easier!

But this is all on Brad’s blog post. I don’t need to review that for you. What I want to show is how this makes continuous integration testing on multiple CFML engines a breeze.

I’ve used CommandBox to run my continuous integration (hereafter CI) tests on Travis for a while now. It was super simple to use the testbox run command to run my tests and search for any failed tests. The one drawback with this approach, though, is that CommandBox only runs Lucee 4.5. This was okay for certain projects where the app would only be ran in Lucee, but it didn’t work well for libraries and modules.

For libraries and modules, I mostly turned to CFML-CI, a template to get multi-engine CI running on Travis using Ant. Take a look at that build.xml file, though:

(Prepare to scroll…)

😳

Hard to grok, isn’t it? Up until now, it was arguably the best way to run multiple engines.

Enter CommandBox 3.1.0.

Everything can be done from the .travis.yml file. Here’s the one I use as my template:

Up in the matrix section, I can define all the platforms to run my tests against. In the before_script section, I use the platform to start a server. Then, my simple script mentioned earlier runs my tests for me. Travis will run these engines concurrently, so I can test all my engines in the time it would take to test one.

Here’s what it looks like on Travis:

Multi-Engine CI in Travis

It’s. That. Easy.

Huge props to Denny Valiant and Brad Wood for the amazing work here. Because of this, I can better support Adobe ColdFusion on my modules, something I was constantly reacting to instead of catching with my tests. My CI set up has never been so easy.

Multi-Engine CI in Travis with CommandBox 3.1.0

When does `BeforeAll` run in TestBox?

Just a quick gotcha I found tonight.

I was trying to get Selenium running in a project using Ortus Solution’s fork of CFSelenium. Here’s the test for the login page:

I thought it would be neat and clean to write the setup for the tests after the given and when sections. Only one problem.

Cannot find Selenium in variables

This was confusing to me. Dumping the variables scope in the beforeAll method showed Selenium, but not in my when block.

Turns out this is due to how TestBox packages specs and when it runs its lifecycle methods like beforeAll. The new spec keywords — feature, story, scenario, given, and when — are just aliases to describe. describe blocks are executed before the test lifecycle in order to find all the actual specs — the it and then blocks. That’s why Selenium was not available there.

Moving all the login to the then block fixed this up nice and quick:

An opportunity might exist here for a Pull Request to instead run the beforeAll before any describe blocks, but I’m not sure if it matters all that much. For now, I’m content to just be aware of the fact and start testing with Selenium.

When does `BeforeAll` run in TestBox?

Looping over Structs passed through the Arguments scope

This example came up while I was trying to refactor a for loop in to a StructEach. I was getting a weird error:

can't cast [orderNumber] string to a number value

All I was getting from the TagContext was a reference to the closure of the StructEach. After trying it a few different ways, I decided to turn to the trusty TryCF.com site to see what the issue was and if it was different across the various engines.

So, here’s our example snippet:

I’ve included a “Dump by passing struct directly:” section to compare the different results to what I would have hoped to see.

Let’s look at the results in Lucee:

TryCF Results of StructEach experiment

Yikes! The entire struct gets passed in as the key and the value is set to the index in the arguments array. You can imagine that errors out pretty quickly as you try to use the key as a string. (I was under the impression that the key had to be a string.)

As if this weren’t bad enough, Adobe ColdFusion implements it just a bit differently.

TryCF Results of StructEach experiment

First off, ACF11 doesn’t pass the entire array as the third parameter to the closure. 👎

Next, we see that — when passing the struct through the arguments of another function — instead of passing the filter name (ORDERNUMBER) as the key and the array of order numbers as the value, it passes the position as the key (1) and the struct as the value.

There’s one more example I want to cover. A simple for loop. You might be hoping this one acts better.

You’d be disappointed. 😞

Here’s the code:

Here’s the results from TryCF

For Lucee:

TryCF Results of for loop experiment

And ACF 11:

TryCF Results of for loop experiment

Still not what I would hope for, but at least it’s consistent. ¯\_(ツ)_/¯ (In fact, ACF is consistent between the for loop and the structEach loop. Good for them.)

As far as I can tell, this is due to the (strange, to me) handling of the arguments scope as both indexed and named. The easy way to solve this is…don’t pass the arguments scope containing a struct something you need to loop over that struct!

Also, I got to be honest, I’m not even sure where I’d start with reporting bugs on the different platforms. What are bugs and what aren’t? Which platform is doing it right (or at least less wrong)?

So I’m not going to touch that part; I’m going to stick to just making you aware of the pitfalls with using the arguments scope and looping over structs.

Looping over Structs passed through the Arguments scope

Team Pull Requests

We had a fun activity today as a team at O.C. Tanner. Every Friday we try to take an opportunity to train as a team. Sometimes we train individually with screencast sites like Code School. Lately we’ve been taking turns researching a topic and then teaching it to the others (like “An In Depth Intro to Arrays” or “SQL Tips”). Today, we tried a different activity — we did a training on pull requests by each submitting a pull request to cfdocs.org.

cfdocs.org is an open source website dedicated to quick access to cross-platform docs on cfml tags and functions. Highly recommended.

This isn’t a blog post about submitting pull requests (though you can read a great one here or here). This is to promote the idea that you can both (1) train your team and (2) give back to the community at the same time. I have found that my programming skills have grown in leaps and bounds even in the short time since I started really involving myself in the cfml community. Now the same can be said for our team.

The activity was a great success. Seven (7) pull requests total. Try it with your team today.

Team Pull Requests

Server Toggling in CommandBox

Really quick post here.

I was wanting to set up aliases for CommandBox. Here’s a few examples:

Notice Line 2? The problem was that I wanted to use bss for both box server start and box server stop. “Wouldn’t it be nice,” I wondered, “if CommandBox would just choose the right one?”

Yes. Yes, it would.

I started with a pull request. Brad Wood helped me out with some code review and helped me realize that this was better off as a ForgeBox command.

Brad also talked about how CommandBox commands are moving to modules in the next version.
While I’m super excited for that (Modules are the best!), I wanted this now, so this is a normal CommandBox command.

The code was pretty simple:

The new toggle command is nested in a server folder. CommandBox nests commands in namespaces according to their folder structure. By nesting toggle.cfc in server, I am essentially adding the toggle command to the existing server namespace.

Also note the aliases metadata on the command of toggle. That let’s us just type toggle as opposed to server toggle.

Here’s the box.json:

One thing to note is Line 6. The value in createPackageDirectory overrides the default name for the package. Since my package’s name is server-toggle, CommandBox was installing it to a server-toggle folder. Specifiying server in the createPackageDirectory field ensures this command is under the server namespace.

That’s it! You can check out the code on GitHub or install the package yourself by running install server-toggle in CommandBox.

Server Toggling in CommandBox