698 lines
21 KiB
Plaintext
698 lines
21 KiB
Plaintext
=pod
|
|
|
|
=for comment
|
|
DO NOT EDIT. This Pod was generated by Swim v0.1.46.
|
|
See http://github.com/ingydotnet/swim-pm#readme
|
|
|
|
=encoding utf8
|
|
|
|
=head1 NAME
|
|
|
|
Test::Base - A Data Driven Testing Framework
|
|
|
|
=for html
|
|
<a href="https://travis-ci.org/ingydotnet/test-base-pm"><img src="https://travis-ci.org/ingydotnet/test-base-pm.png" alt="test-base-pm"></a>
|
|
<a href="https://coveralls.io/r/ingydotnet/test-base-pm?branch=master"><img src="https://coveralls.io/repos/ingydotnet/test-base-pm/badge.png" alt="test-base-pm"></a>
|
|
|
|
=head1 SYNOPSIS
|
|
|
|
A new test module:
|
|
|
|
# lib/MyProject/Test.pm
|
|
package MyProject::Test;
|
|
use Test::Base -Base;
|
|
|
|
use MyProject;
|
|
|
|
package MyProject::Test::Filter;
|
|
use Test::Base::Filter -base;
|
|
|
|
sub my_filter {
|
|
return MyProject->do_something(shift);
|
|
}
|
|
|
|
A sample test:
|
|
|
|
# t/sample.t
|
|
use MyProject::Test;
|
|
|
|
plan tests => 1 * blocks;
|
|
|
|
run_is input => 'expected';
|
|
|
|
sub local_filter {
|
|
s/my/your/;
|
|
}
|
|
|
|
__END__
|
|
|
|
=== Test one (the name of the test)
|
|
--- input my_filter local_filter
|
|
my
|
|
input
|
|
lines
|
|
--- expected
|
|
expected
|
|
output
|
|
|
|
=== Test two
|
|
This is an optional description
|
|
of this particular test.
|
|
--- input my_filter
|
|
other
|
|
input
|
|
lines
|
|
--- expected
|
|
other expected
|
|
output
|
|
|
|
=head1 DESCRIPTION
|
|
|
|
Testing is usually the ugly part of Perl module authoring. Perl gives you a
|
|
standard way to run tests with Test::Harness, and basic testing primitives
|
|
with Test::More. After that you are pretty much on your own to develop a
|
|
testing framework and philosophy. Test::More encourages you to make your own
|
|
framework by subclassing Test::Builder, but that is not trivial.
|
|
|
|
Test::Base gives you a way to write your own test framework base class that
|
|
I<is> trivial. In fact it is as simple as two lines:
|
|
|
|
package MyTestFramework;
|
|
use Test::Base -Base;
|
|
|
|
A module called C<MyTestFramework.pm> containing those two lines, will give
|
|
all the power of Test::More and all the power of Test::Base to every test file
|
|
that uses it. As you build up the capabilities of C<MyTestFramework>, your
|
|
tests will have all of that power as well.
|
|
|
|
C<MyTestFramework> becomes a place for you to put all of your reusable testing
|
|
bits. As you write tests, you will see patterns and duplication, and you can
|
|
"upstream" them into C<MyTestFramework>. Of course, you don't have to subclass
|
|
Test::Base at all. You can use it directly in many applications, including
|
|
everywhere you would use Test::More.
|
|
|
|
Test::Base concentrates on offering reusable data driven patterns, so that you
|
|
can write tests with a minimum of code. At the heart of all testing you have
|
|
inputs, processes and expected outputs. Test::Base provides some clean ways
|
|
for you to express your input and expected output data, so you can spend your
|
|
|
|
time focusing on that rather than your code scaffolding.
|
|
|
|
=head1 EXPORTED FUNCTIONS
|
|
|
|
Test::Base extends Test::More and exports all of its functions. So you can
|
|
basically write your tests the same as Test::More. Test::Base also exports
|
|
many functions of its own:
|
|
|
|
=over
|
|
|
|
=item C<is(actual, expected, [test-name])>
|
|
|
|
This is the equivalent of Test::More's C<is> function with one interesting
|
|
twist. If your actual and expected results differ and the output is multi-
|
|
line, this function will show you a unified diff format of output. Consider
|
|
the benefit when looking for the one character that is different in hundreds
|
|
of lines of output!
|
|
|
|
Diff output requires the optional C<Text::Diff> CPAN module. If you don't have
|
|
this module, the C<is()> function will simply give you normal Test::More
|
|
output. To disable diffing altogether, set the C<TEST_SHOW_NO_DIFFS>
|
|
environment variable (or C<$ENV{TEST_SHOW_NO_DIFFS}>) to a true value. You can
|
|
also call the C<no_diff> function as a shortcut.
|
|
|
|
=item C<blocks( [data-section-name] )>
|
|
|
|
The most important function is C<blocks>. In list context it returns a list of
|
|
C<Test::Base::Block> objects that are generated from the test specification in
|
|
the C<DATA> section of your test file. In scalar context it returns the number
|
|
of objects. This is useful to calculate your Test::More plan.
|
|
|
|
Each Test::Base::Block object has methods that correspond to the names of that
|
|
object's data sections. There is also a C<name> and a C<description> method
|
|
for accessing those parts of the block if they were specified.
|
|
|
|
The C<blocks> function can take an optional single argument, that indicates to
|
|
only return the blocks that contain a particular named data section. Otherwise
|
|
C<blocks> returns all blocks.
|
|
|
|
my @all_of_my_blocks = blocks;
|
|
|
|
my @just_the_foo_blocks = blocks('foo');
|
|
|
|
=item C<next_block()>
|
|
|
|
You can use the next_block function to iterate over all the blocks.
|
|
|
|
while (my $block = next_block) {
|
|
...
|
|
}
|
|
|
|
It returns undef after all blocks have been iterated over. It can then be
|
|
called again to reiterate.
|
|
|
|
=item C<first_block()>
|
|
|
|
Returns the first block or undef if there are none. It resets the iterator to
|
|
the C<next_block> function.
|
|
|
|
=item C<run(&subroutine)>
|
|
|
|
There are many ways to write your tests. You can reference each block
|
|
individually or you can loop over all the blocks and perform a common
|
|
operation. The C<run> function does the looping for you, so all you need to do
|
|
is pass it a code block to execute for each block.
|
|
|
|
The C<run> function takes a subroutine as an argument, and calls the sub one
|
|
time for each block in the specification. It passes the current block object
|
|
to the subroutine.
|
|
|
|
run {
|
|
my $block = shift;
|
|
is(process($block->foo), $block->bar, $block->name);
|
|
};
|
|
|
|
=item C<run_is([data_name1, data_name2])>
|
|
|
|
Many times you simply want to see if two data sections are equivalent in
|
|
every block, probably after having been run through one or more filters. With
|
|
the C<run_is> function, you can just pass the names of any two data sections
|
|
that exist in every block, and it will loop over every block comparing the
|
|
two sections.
|
|
|
|
run_is 'foo', 'bar';
|
|
|
|
If no data sections are given C<run_is> will try to detect them automatically.
|
|
|
|
NOTE: Test::Base will silently ignore any blocks that don't contain
|
|
both sections.
|
|
|
|
=item C<is_deep($data1, $data2, $test_name)>
|
|
|
|
Like Test::More's C<is_deeply> but uses the more correct Test::Deep module.
|
|
|
|
=item C<run_is_deeply([data_name1, data_name2])>
|
|
|
|
Like C<run_is_deeply> but uses C<is_deep> which uses the more correct
|
|
Test::Deep.
|
|
|
|
=item C<run_is_deeply([data_name1, data_name2])>
|
|
|
|
Like C<run_is> but uses C<is_deeply> for complex data structure comparison.
|
|
|
|
=item C<run_is_deeply([data_name1, data_name2])>
|
|
|
|
Like C<run_is_deeply> but uses C<is_deep> which uses the more correct
|
|
Test::Deep.
|
|
|
|
=item C<run_like([data_name, regexp | data_name]);>
|
|
|
|
The C<run_like> function is similar to C<run_is> except the second argument is
|
|
a regular expression. The regexp can either be a C<qr{}> object or a data
|
|
section that has been filtered into a regular expression.
|
|
|
|
run_like 'foo', qr{<html.*};
|
|
run_like 'foo', 'match';
|
|
|
|
=item C<run_unlike([data_name, regexp | data_name]);>
|
|
|
|
The C<run_unlike> function is similar to C<run_like>, except the opposite.
|
|
|
|
run_unlike 'foo', qr{<html.*};
|
|
run_unlike 'foo', 'no_match';
|
|
|
|
=item C<run_compare(data_name1, data_name2)>
|
|
|
|
The C<run_compare> function is like the C<run_is>, C<run_is_deeply> and the
|
|
C<run_like> functions all rolled into one. It loops over each relevant block
|
|
and determines what type of comparison to do.
|
|
|
|
NOTE: If you do not specify either a plan, or run any tests, the
|
|
C<run_compare> function will automatically be run.
|
|
|
|
=item C<delimiters($block_delimiter, $data_delimiter)>
|
|
|
|
Override the default delimiters of C<===> and C<--->.
|
|
|
|
=item C<spec_file($file_name)>
|
|
|
|
By default, Test::Base reads its input from the DATA section. This function
|
|
tells it to get the spec from a file instead.
|
|
|
|
=item C<spec_string($test_data)>
|
|
|
|
By default, Test::Base reads its input from the DATA section. This function
|
|
tells it to get the spec from a string that has been prepared somehow.
|
|
|
|
=item C<filters( @filters_list or $filters_hashref )>
|
|
|
|
Specify a list of additional filters to be applied to all blocks. See
|
|
C<FILTERS> below.
|
|
|
|
You can also specify a hash ref that maps data section names to an array ref
|
|
of filters for that data type.
|
|
|
|
filters {
|
|
xxx => [qw(chomp lines)],
|
|
yyy => ['yaml'],
|
|
zzz => 'eval',
|
|
};
|
|
|
|
If a filters list has only one element, the array ref is optional.
|
|
|
|
=item C<filters_delay( [1 | 0] );>
|
|
|
|
By default Test::Base::Block objects are have all their filters run ahead of
|
|
time. There are testing situations in which it is advantageous to delay the
|
|
filtering. Calling this function with no arguments or a true value, causes the
|
|
filtering to be delayed.
|
|
|
|
use Test::Base;
|
|
filters_delay;
|
|
plan tests => 1 * blocks;
|
|
for my $block (blocks) {
|
|
...
|
|
$block->run_filters;
|
|
ok($block->is_filtered);
|
|
...
|
|
}
|
|
|
|
In the code above, the filters are called manually, using the C<run_filters>
|
|
method of Test::Base::Block. In functions like C<run_is>, where the tests are
|
|
run automatically, filtering is delayed until right before the test.
|
|
|
|
=item C<filter_arguments()>
|
|
|
|
Return the arguments after the equals sign on a filter.
|
|
|
|
sub my_filter {
|
|
my $args = filter_arguments;
|
|
# is($args, 'whazzup');
|
|
...
|
|
}
|
|
|
|
__DATA__
|
|
=== A test
|
|
--- data my_filter=whazzup
|
|
|
|
=item C<tie_output()>
|
|
|
|
You can capture STDOUT and STDERR for operations with this function:
|
|
|
|
my $out = '';
|
|
tie_output(*STDOUT, $out);
|
|
print "Hey!\n";
|
|
print "Che!\n";
|
|
untie *STDOUT;
|
|
is($out, "Hey!\nChe!\n");
|
|
|
|
=item C<no_diff()>
|
|
|
|
Turn off diff support for is() in a test file.
|
|
|
|
=item C<default_object()>
|
|
|
|
Returns the default Test::Base object. This is useful if you feel the need to
|
|
do an OO operation in otherwise functional test code. See L<OO> below.
|
|
|
|
=item C<WWW() XXX() YYY() ZZZ()>
|
|
|
|
These debugging functions are exported from the Spiffy.pm module. See
|
|
L<Spiffy> for more info.
|
|
|
|
=item C<croak() carp() cluck() confess()>
|
|
|
|
You can use the functions from the Carp module without needing to import them.
|
|
Test::Base does it for you by default.
|
|
|
|
=back
|
|
|
|
=head1 TEST SPECIFICATION
|
|
|
|
Test::Base allows you to specify your test data in an external file, the
|
|
DATA section of your program or from a scalar variable containing all the
|
|
text input.
|
|
|
|
A I<test specification> is a series of text lines. Each test (or block) is
|
|
separated by a line containing the block delimiter and an optional test
|
|
C<name>. Each block is further subdivided into named sections with a line
|
|
containing the data delimiter and the data section name. A C<description>
|
|
of the test can go on lines after the block delimiter but before the first
|
|
data section.
|
|
|
|
Here is the basic layout of a specification:
|
|
|
|
=== <block name 1>
|
|
<optional block description lines>
|
|
--- <data section name 1> <filter-1> <filter-2> <filter-n>
|
|
<test data lines>
|
|
--- <data section name 2> <filter-1> <filter-2> <filter-n>
|
|
<test data lines>
|
|
--- <data section name n> <filter-1> <filter-2> <filter-n>
|
|
<test data lines>
|
|
|
|
=== <block name 2>
|
|
<optional block description lines>
|
|
--- <data section name 1> <filter-1> <filter-2> <filter-n>
|
|
<test data lines>
|
|
--- <data section name 2> <filter-1> <filter-2> <filter-n>
|
|
<test data lines>
|
|
--- <data section name n> <filter-1> <filter-2> <filter-n>
|
|
<test data lines>
|
|
|
|
Here is a code example:
|
|
|
|
use Test::Base;
|
|
|
|
delimiters qw(### :::);
|
|
|
|
# test code here
|
|
|
|
__END__
|
|
|
|
### Test One
|
|
We want to see if foo and bar
|
|
are really the same...
|
|
::: foo
|
|
a foo line
|
|
another foo line
|
|
|
|
::: bar
|
|
a bar line
|
|
another bar line
|
|
|
|
### Test Two
|
|
|
|
::: foo
|
|
some foo line
|
|
some other foo line
|
|
|
|
::: bar
|
|
some bar line
|
|
some other bar line
|
|
|
|
::: baz
|
|
some baz line
|
|
some other baz line
|
|
|
|
This example specifies two blocks. They both have foo and bar data sections.
|
|
The second block has a baz component. The block delimiter is C<###> and the
|
|
data delimiter is C<:::>.
|
|
|
|
The default block delimiter is C<===> and the default data delimiter is C<---
|
|
>.
|
|
|
|
There are some special data section names used for control purposes:
|
|
|
|
--- SKIP
|
|
--- ONLY
|
|
--- LAST
|
|
|
|
A block with a SKIP section causes that test to be ignored. This is useful to
|
|
disable a test temporarily.
|
|
|
|
A block with an ONLY section causes only that block to be used. This is useful
|
|
when you are concentrating on getting a single test to pass. If there is more
|
|
than one block with ONLY, the first one will be chosen.
|
|
|
|
Because ONLY is very useful for debugging and sometimes you forgot to remove
|
|
the ONLY flag before committing to the VCS or uploading to CPAN, Test::Base by
|
|
default gives you a diag message saying I<I found ONLY ... maybe you're
|
|
debugging?>. If you don't like it, use C<no_diag_on_only>.
|
|
|
|
A block with a LAST section makes that block the last one in the
|
|
specification. All following blocks will be ignored.
|
|
|
|
=head1 FILTERS
|
|
|
|
The real power in writing tests with Test::Base comes from its filtering
|
|
capabilities. Test::Base comes with an ever growing set of useful generic
|
|
filters than you can sequence and apply to various test blocks. That means you
|
|
can specify the block serialization in the most readable format you can find,
|
|
and let the filters translate it into what you really need for a test. It is
|
|
easy to write your own filters as well.
|
|
|
|
Test::Base allows you to specify a list of filters to each data section of
|
|
each block. The default filters are C<norm> and C<trim>. These filters will be
|
|
applied (in order) to the data after it has been parsed from the specification
|
|
and before it is set into its Test::Base::Block object.
|
|
|
|
You can add to the default filter list with the C<filters> function. You can
|
|
specify additional filters to a specific block by listing them after the
|
|
section name on a data section delimiter line.
|
|
|
|
Example:
|
|
|
|
use Test::Base;
|
|
|
|
filters qw(foo bar);
|
|
filters { perl => 'strict' };
|
|
|
|
sub upper { uc(shift) }
|
|
|
|
__END__
|
|
|
|
=== Test one
|
|
--- foo trim chomp upper
|
|
...
|
|
|
|
--- bar -norm
|
|
...
|
|
|
|
--- perl eval dumper
|
|
my @foo = map {
|
|
- $_;
|
|
} 1..10;
|
|
\ @foo;
|
|
|
|
Putting a C<-> before a filter on a delimiter line, disables that filter.
|
|
|
|
=head2 Scalar vs List
|
|
|
|
Each filter can take either a scalar or a list as input, and will return
|
|
either a scalar or a list. Since filters are chained together, it is
|
|
important to learn which filters expect which kind of input and return which
|
|
kind of output.
|
|
|
|
For example, consider the following filter list:
|
|
|
|
norm trim lines chomp array dumper eval
|
|
|
|
The data always starts out as a single scalar string. C<norm> takes a scalar
|
|
and returns a scalar. C<trim> takes a list and returns a list, but a scalar is
|
|
a valid list. C<lines> takes a scalar and returns a list. C<chomp> takes a
|
|
list and returns a list. C<array> takes a list and returns a scalar (an
|
|
anonymous array reference containing the list elements). C<dumper> takes a
|
|
list and returns a scalar. C<eval> takes a scalar and creates a list.
|
|
|
|
A list of exactly one element works fine as input to a filter requiring a
|
|
scalar, but any other list will cause an exception. A scalar in list context
|
|
is considered a list of one element.
|
|
|
|
Data accessor methods for blocks will return a list of values when used in
|
|
list context, and the first element of the list in scalar context. This is
|
|
usually "the right thing", but be aware.
|
|
|
|
=head2 The Stock Filters
|
|
|
|
Test::Base comes with large set of stock filters. They are in the
|
|
C<Test::Base::Filter> module. See L<Test::Base::Filter> for a listing and
|
|
description of these filters.
|
|
|
|
=head2 Rolling Your Own Filters
|
|
|
|
Creating filter extensions is very simple. You can either write a I<function>
|
|
in the C<main> namespace, or a I<method> in the C<Test::Base::Filter>
|
|
namespace or a subclass of it. In either case the text and any extra arguments
|
|
are passed in and you return whatever you want the new value to be.
|
|
|
|
Here is a self explanatory example:
|
|
|
|
use Test::Base;
|
|
|
|
filters 'foo', 'bar=xyz';
|
|
|
|
sub foo {
|
|
transform(shift);
|
|
}
|
|
|
|
sub Test::Base::Filter::bar {
|
|
my $self = shift; # The Test::Base::Filter object
|
|
my $data = shift;
|
|
my $args = $self->current_arguments;
|
|
my $current_block_object = $self->block;
|
|
# transform $data in a barish manner
|
|
return $data;
|
|
}
|
|
|
|
If you use the method interface for a filter, you can access the block
|
|
internals by calling the C<block> method on the filter object.
|
|
|
|
Normally you'll probably just use the functional interface, although all the
|
|
builtin filters are methods.
|
|
|
|
Note that filters defined in the C<main> namespace can look like:
|
|
|
|
sub filter9 {
|
|
s/foo/bar/;
|
|
}
|
|
|
|
since Test::Base automatically munges the input string into $_ variable and
|
|
checks the return value of the function to see if it looks like a number.
|
|
If you must define a filter that returns just a single number, do it in a
|
|
different namespace as a method. These filters don't allow the simplistic
|
|
$_ munging.
|
|
|
|
=head1 OO
|
|
|
|
Test::Base has a nice functional interface for simple usage. Under the hood
|
|
everything is object oriented. A default Test::Base object is created and all
|
|
the functions are really just method calls on it.
|
|
|
|
This means if you need to get fancy, you can use all the object oriented stuff
|
|
too. Just create new Test::Base objects and use the functions as methods.
|
|
|
|
use Test::Base;
|
|
my $blocks1 = Test::Base->new;
|
|
my $blocks2 = Test::Base->new;
|
|
|
|
$blocks1->delimiters(qw(!!! @@@))->spec_file('test1.txt');
|
|
$blocks2->delimiters(qw(### $$$))->spec_string($test_data);
|
|
|
|
plan tests => $blocks1->blocks + $blocks2->blocks;
|
|
|
|
# ... etc
|
|
|
|
=head1 THE C<TEST::BASE::BLOCK> CLASS
|
|
|
|
In Test::Base, blocks are exposed as Test::Base::Block objects. This section
|
|
lists the methods that can be called on a Test::Base::Block object. Of course,
|
|
each data section name is also available as a method.
|
|
|
|
=over
|
|
|
|
=item C<name()>
|
|
|
|
This is the optional short description of a block, that is specified on the
|
|
block separator line.
|
|
|
|
=item C<description()>
|
|
|
|
This is an optional long description of the block. It is the text taken from
|
|
between the block separator and the first data section.
|
|
|
|
=item C<seq_num()>
|
|
|
|
Returns a sequence number for this block. Sequence numbers begin with 1.
|
|
|
|
=item C<blocks_object()>
|
|
|
|
Returns the Test::Base object that owns this block.
|
|
|
|
=item C<run_filters()>
|
|
|
|
Run the filters on the data sections of the blocks. You don't need to use this
|
|
method unless you also used the C<filters_delay> function.
|
|
|
|
=item C<is_filtered()>
|
|
|
|
Returns true if filters have already been run for this block.
|
|
|
|
=item C<original_values()>
|
|
|
|
Returns a hash of the original, unfiltered values of each data section.
|
|
|
|
=back
|
|
|
|
=head1 SUBCLASSING
|
|
|
|
One of the nicest things about Test::Base is that it is easy to subclass. This
|
|
is very important, because in your personal project, you will likely want to
|
|
extend Test::Base with your own filters and other reusable pieces of your test
|
|
framework.
|
|
|
|
Here is an example of a subclass:
|
|
|
|
package MyTestStuff;
|
|
use Test::Base -Base;
|
|
|
|
our @EXPORT = qw(some_func);
|
|
|
|
sub some_func {
|
|
(my ($self), @_) = find_my_self(@_);
|
|
...
|
|
}
|
|
|
|
package MyTestStuff::Block;
|
|
use base 'Test::Base::Block';
|
|
|
|
sub desc {
|
|
$self->description(@_);
|
|
}
|
|
|
|
package MyTestStuff::Filter;
|
|
use base 'Test::Base::Filter';
|
|
|
|
sub upper {
|
|
$self->assert_scalar(@_);
|
|
uc(shift);
|
|
}
|
|
|
|
Note that you don't have to re-Export all the functions from Test::Base. That
|
|
happens automatically, due to the powers of Spiffy.
|
|
|
|
The first line in C<some_func> allows it to be called as either a function or
|
|
a method in the test code.
|
|
|
|
=head1 DISTRIBUTION SUPPORT
|
|
|
|
You might be thinking that you do not want to use Test::Base in you modules,
|
|
because it adds an installation dependency. Fear not.
|
|
L<Module::Install::TestBase> takes care of that.
|
|
|
|
Just write a Makefile.PL that looks something like this:
|
|
|
|
use inc::Module::Install;
|
|
|
|
name 'Foo';
|
|
all_from 'lib/Foo.pm';
|
|
|
|
use_test_base;
|
|
|
|
WriteAll;
|
|
|
|
The line with C<use_test_base> will automatically bundle all the code the user
|
|
needs to run Test::Base based tests.
|
|
|
|
=head1 OTHER COOL FEATURES
|
|
|
|
Test::Base automatically adds:
|
|
|
|
use strict;
|
|
use warnings;
|
|
|
|
to all of your test scripts and Test::Base subclasses. A Spiffy feature
|
|
indeed.
|
|
|
|
=head1 HISTORY
|
|
|
|
This module started its life with the horrible and ridicule inducing name
|
|
C<Test::Chunks>. It was renamed to C<Test::Base> with the hope that it would
|
|
be seen for the very useful module that it has become. If you are switching
|
|
from C<Test::Chunks> to C<Test::Base>, simply substitute the concept and usage
|
|
of C<chunks> to C<blocks>.
|
|
|
|
=head1 AUTHOR
|
|
|
|
Ingy döt Net <ingy@cpan.org>
|
|
|
|
=head1 COPYRIGHT
|
|
|
|
Copyright 2005-2018. Ingy döt Net.
|
|
|
|
This program is free software; you can redistribute it and/or modify it under
|
|
the same terms as Perl itself.
|
|
|
|
See L<http://www.perl.com/perl/misc/Artistic.html>
|
|
|
|
=cut
|