Wednesday, May 6, 2009

Using KiokuDB in Catalyst applications

Using KiokuDB with Catalyst is very easy. This article sums up a few lessons learned from the last several apps we've developed at my workplace, and introduces the modules we refactored out of them.

Let's write an app is called Kitten::Friend, in which kittens partake in a social network and upload pictures of vases they've broken.

We generally follow these rules for organizing our code:

  • Catalyst stuff goes under Kitten::Friend::Web::.
  • Reusable app model code goes under Kitten::Friend::Model::.
  • The actual domain objects go under Kitten::Friend::Schema::, for instance Kitten::Friend::Schema::Vase. Using DBIx::Class::Schema these would be the table classes.

Anything that is not dependent on the Catalyst environment (as much code as possible) is kept separate from it. This means that we can use our KiokuDB model with all the convenience methods for unit testing or scripts, without configuring the Catalyst specific bits.

Functionality relating to how the app actually behaves is put in the Schema namespace. We try to keep this code quite pure.

Glue code and helper methods go in the Model namespace. This separation helps us to refactor and adapt the code quite easily.

So let's start with the schema objects. Let's say we have two classes. The first is Kitten::Friend::Schema::Kitten:

package Kitten::Friend::Schema::Kitten;
use Moose;

use Kitten::Friend::Schema::Vase;

use KiokuDB::Set;

use KiokuDB::Util qw(set);

use namespace::autoclean;

with qw(KiokuX::User);    # provides 'id' and 'password' attributes

has name => (
    isa      => "Str",
    is       => "ro",
    required => 1,

has friends => (
    isa     => "KiokuDB::Set",
    is      => "ro",
    lazy    => 1,
    default => sub { set() },    # empty set

has vases => (
    isa     => "KiokuDB::Set",
    is      => "ro",
    lazy    => 1,
    default => sub { set() },

sub new_vase {
    my ( $self, @args ) = @_;

    my $vase = Kitten::Friend::Schema::Vase->new(
        owner => $self,


    return $vase;

sub add_friend {
    my ( $self, $friend ) = @_;



I've used the KiokuX::User role to provide the Kitten object with some standard attributes for user objects. This will be used later to provide authentication support.

The second class is Kitten::Friend::Schema::Vase:

package Kitten::Friend::Schema::Vase;
use Moose;

use MooseX::AttributeHelpers;
use URI;

use namespace::autoclean;

has pictures => (
    metaclass => "Collection::Array",
    isa       => "ArrayRef[URI]",
    is        => "ro",
    default   => sub { [] },
    provides  => {
        push => "add_picture",

has owner => (
    isa      => "Kitten::Friend::Schema::Kitten",
    is       => "ro",
    required => 1,


Now let's write a unit test:

use strict;
use warnings;

use Test::More 'no_plan';

use KiokuX::User::Util qw(crypt_password);

use ok 'Kitten::Friend::Schema::Kitten';

my $kitten = Kitten::Friend::Schema::Kitten->new(
    name     => "Snookums",
    id       => "cutesy843",
    password => crypt_password("luvt00na"),

isa_ok( $kitten, "Kitten::Friend::Schema::Kitten" );

is( $kitten->name, "Snookums", "name attribute" );

ok( $kitten->check_password("luvt00na"), "password check" );
ok( !$kitten->check_password("bathtime"), "bad password" );

is_deeply( [ $kitten->friends->members ], [ ], "no friends" );
is_deeply( [ $kitten->vases->members ], [ ], "no no vases" );

my $vase = $kitten->new_vase(
    pictures => [ URI->new("") ],

isa_ok( $vase, "Kitten::Friend::Schema::Vase" );

is_deeply( [ $kitten->vases->members ], [ $vase ], "new vase added" );

This test obviously runs completely independently of either Catalyst or KiokuDB.

The next step is to set up the model. We use KiokuX::Model as our model base class. The model class usually contains helper methods, like txn_do call wrappers and various other storage oriented tasks we want to abstract away. This way the web app code and scripts get a simpler API that takes care of as many persistence details as possible.

package Kitten::Friend::Model::KiokuDB;
use Moose;

extends qw(KiokuX::Model);

sub insert_kitten {
    my ( $self, $kitten ) = @_;

    my $id = $self->txn_do(sub {

    return $id;


We can write a t/model.t to try this out:

use strict;
use warnings;

use Test::More 'no_plan';

use ok 'Kitten::Friend::Model::KiokuDB';

my $m = Kitten::Friend::Model::KiokuDB->new( dsn => "hash" );

    my $s = $m->new_scope;

    my $id = $m->insert_kitten(
            name     => "Kitteh",
            id       => "kitteh",
            password => crypt_password("s33krit"),

    ok( $id, "got an ID" );

    my $kitten = $m->lookup($id);

    isa_ok( $kitten, "Kitten::Friend::Schema::Kitten", "looked up object" );

Next up is gluing this into the Catalyst app itself. I'm assuming you generated the app structure with Kitten::Friend::Web. Create Kitten::Friend::Web::Model::KiokuDB as a subclass of Catalyst::Model::KiokuDB:

package Kitten::Friend::Web::Model::KiokuDB;
use Moose;

use Kitten::Friend::Model::KiokuDB;

BEGIN { extends qw(Catalyst::Model::KiokuDB) }

has '+model_class' => ( default => "Kitten::Friend::Model::KiokuDB" );

And then configure a DSN in your or configuration file:
>Model KiokuDB<
    dsn dbi:SQLite:dbname=root/db

That's all that's necessary to glue our Catalyst independent model code into the web app part.

Your model methods can be called as:

my $m = $c->model("kiokudb");

my $inner = $m->model


In the future I might consider adding an AUTOLOAD method, but you can also just extend the model attribute of Catalyst::Model::KiokuDB to provide more delegations (currently it only delegates KiokuDB::Role::API).

If you'd like to use Catalyst::Plugin::Authentication, configure it as follows:

    'Plugin::Authentication' => {
        realms => {
            default => {
                credential => {
                    class         => 'Password',
                    password_type => 'self_check'
                store => {
                    class      => 'Model::KiokuDB',
                    model_name => "kiokudb",

And then you can let your kittens log in to the website:

my $user = eval {
        id       => $id,
        password => $password,

if ( $user ) {
    $c->response->body( "Hello " . $user->get_object->name )

Some bonus features of the Catalyst model:

  • It calls new_scope for you, once per request
  • It tracks leaked objects and reports them in Debug mode. Circular structures are a bit tricky to get right if you aren't used to them so this is a big help.


Unknown said...

Great post and shows some of rapid development options with modern Perl. My question for you is if you could have used MooseX::Types and if KiokuDB is compatible with MooseX::Declare?


nothingmuch said...

Yes, I could have definitely used both!.

I thought that MX::Declare might be a bit distracting to people who don't know it yet. The code as is is quite readable I think and I wanted to focus on the KiokuDB specific bits.

Similarly, the only case for MX::Types involves the 'URI' class type, and digressing into why bareword class names are annoying doesn't seem helpful.

These minor issues aside, I whole heartedly recommend using both MooseX::Types and MooseX::Declare, I love both modules!

If you want MX::Declare classes with the KiokuDB::Lazy trait just make sure the metaclass is set up correctly (more info in the KiokuDB::Class docs)

Mark said...

Thanks for the detailed post. This is very interesting and clears up many of my question about KiokuDB and even a few about how Moose works in Catalyst v5.8.

My nagging question revolves around the search aspect of an OODB. Just thinking through one of the common searches (without going into anything even approaching data mining which I know can be problematic for OODBs) how would you efficiently handle something like a password reset? Say a user gives us their email address, should I search through every User object for the address? Should I use dbi and have a column for each email address, even though most of my objects won't have one (and I can envision potential issues will pulling an email address from the wrong type of object)? Or should I address this with Search::GIN? I think any of these would work, but as I understand them, all have their draw backs. What is the "best" way to deal with something like this.


nothingmuch said...

The current best way is to either use the DBI backend's search columns, or Search::GIN::Extract::Callback and Search::GIN::Query::Manual. I will write a short post on the latter soon, but I think it's also covered in the tutorial.

If you want to use the extra columns I wouldn't worry about the column, having a NULL value is, after all, what NULL values were designed for ;-)

The problem with Search::GIN is that you can't easily compose multiple indexes yet, and the query execution is very low level at the moment (the Query objects are really more like opcodes in a query language's execution optree).

However, if you constraint yourself to Callback and Query::Manual things are pretty flexible. it's not what Search::GIN was designed to do in the long term but it's useful right now.

I'm off to bed now, but feel free to come to #kiokudb on IRC and I can paste a few examples tomorrow morning.