Expectations (assertions)

Expectations are pure, composable values. This forces developers to separate the test's checks from the scenario, which is generally cleaner/clearer.

The easiest way to construct expectactions is to call the expect macro. The clue function can be used to investigate failures.

TL;DR

• Assert on boolean values using expect:

  expect(list.size > 4)

• Investigate failures using clue:

  expect(clue(list).size > 4)

• Compare values using expect.eql (needs an Eq instance)

  expect.eql(List(1, 2, 3), (1 to 3).toList)

• Compare values with relaxed equality using expect.same (no Eq instance needed)

  expect.same(List(1, 2, 3), (1 to 3).toList)

• Compose expectations using and/or

  (expect(1 > 0) and expect(2 > 1)) or expect(5 < 6)

• ... or their symbolic aliases &&/||

  (expect(1 > 0) && expect(2 > 1)) || expect(5 < 6)

• Compose expectations using xor

  expect(1 > 0) xor expect(2 > 1)

• Use varargs short form for asserting on all boolean values

  expect.all(1 > 0, 2 > 1, 3 > 2)

• Use forEach to test every element of a collection (or anything that implements Foldable)

  forEach(List(1, 2, 3))(i => expect(i < 5))

• Use exists to assert that at least one element of collection matches expectations:

  exists(Option(5))(n => expect(n > 3))

• Use matches to assert that an expression matches a given pattern

  matches(Option(4)) { case Some(x) =>
  expect.eql(4, x)
  }

• Use whenSuccess to assert that a structure with an error channel (like Either) is successful

  val res: Either[String, Int] =
  Right(4)
  
  whenSuccess(res) { n =>
  expect.eql(4, n)
  }

• Use success or failure to create succeeding/failing expectations without conditions

  val result = if(5 == 5) success else failure("oh no")

• Use .failFast to evaluate the expectation eagerly and raise the assertion error in your effect type

  for {
  x <- IO("hello")
  _ <- expect(x.length < 4).failFast[IO]
  y = x + "bla"
  _ <- expect(y.size > x.size).failFast[IO]
  } yield expect(y.contains(x))

Example suite

import weaver._
import cats.effect.IO

object ExpectationsSuite extends SimpleIOSuite {

  def test(a: Int) = a + 5

  pureTest("Simple expectations (success)") {
    val z = 15

    expect(test(z) < z + 6)
  }

  pureTest("Simple expectations (failure)") {
    val z = 15

    expect(clue(test(z)) > z + 6)
  }

  pureTest("And/Or composition (success)") {
    expect(1 != 2) and expect(2 != 1) or expect(2 != 3)
  }

  pureTest("And/Or composition (failure)") {
    (expect(1 != clue(2)) and expect(2 < clue(1))) or expect(2 > clue(3))
  }

  pureTest("Varargs composition (success)") {
    // expect(1 + 1 < 3) && expect(2 + 2 < 5) && expect(4 * 2 < 9)
    expect.all(1 + 1 < 3, 2 + 2 < 5, 4 * 2 < 9)
  }

  pureTest("Varargs composition (failure)") {
    // expect(1 + 1 < 3) && expect(clue(2 + 2) > 5) && expect(4 * 2 < 9)
    expect.all(1 + 1 < 3, clue(2 + 2) > 5, 4 * 2 < 9)
  }

  pureTest("Working with collections (success)") {
    forEach(List(1, 2, 3))(i => expect(i < 5)) and
      forEach(Option("hello"))(msg => expect.eql(msg, "hello")) and
      exists(List("a", "b", "c"))(i => expect.eql(i, "c")) and
      exists(Vector(true, true, false))(i => expect.eql(i, false))
  }

  pureTest("Working with collections (failure 1)") {
    forEach(Vector("hello", "world"))(msg => expect.eql(msg, "hello"))
  }

  pureTest("Working with collections (failure 2)") {
    exists(Option(39))(i => expect(clue(i) > 50))
  }

  import cats.Eq
  case class Test(d: Double)

  implicit val eqTest: Eq[Test] = Eq.by[Test, Double](_.d)

  pureTest("Strict equality (success)") {
    expect.eql("hello", "hello") and
      expect.eql(List(1, 2, 3), List(1, 2, 3)) and
      expect.eql(Test(25.0), Test(25.0))
  }

  pureTest("Strict equality (failure 1)") {
    expect.eql("hello", "world")
  }

  pureTest("Strict equality (failure 2)") {
    expect.eql(List(1, 2, 3), List(1, 19, 3))
  }

  pureTest("Strict equality (failure 3)") {
    expect.eql(Test(25.0), Test(50.0))
  }

  // Note that we don't have an instance of Eq[Hello]
  // anywhere in scope
  class Hello(val d: Double) {
    override def toString = s"Hello to $d"

    override def equals(other: Any) =
      if(other != null && other.isInstanceOf[Hello])
        other.asInstanceOf[Hello].d == this.d
      else
        false
  }

  pureTest("Relaxed equality comparison (success)") {
    expect.same(new Hello(25.0), new Hello(25.0))
  }

  pureTest("Relaxed equality comparison (failure)") {
    expect.same(new Hello(25.0), new Hello(50.0))
  }

  pureTest("Non macro-based expectations") {
    val condition : Boolean = false
    if (condition) success else failure("Condition failed")
  }

  test("Failing fast expectations") {
    for {
      h <- IO.pure("hello")
      _ <- expect(clue(h).isEmpty).failFast
    } yield success
  }
}

Example suite report

repl.MdocSessionMdocAppExpectationsSuite
+ Simple expectations (success) 2ms
+ And/Or composition (success) 12ms
+ Varargs composition (success) 8ms
+ Working with collections (success) 14ms
+ Strict equality (success) 5ms
+ Relaxed equality comparison (success) 5ms
- Simple expectations (failure) 2ms
- And/Or composition (failure) 9ms
- Varargs composition (failure) 8ms
- Working with collections (failure 1) 5ms
- Working with collections (failure 2) 6ms
- Strict equality (failure 1) 0ms
- Strict equality (failure 2) 2ms
- Strict equality (failure 3) 1ms
- Relaxed equality comparison (failure) 2ms
- Non macro-based expectations 0ms
- Failing fast expectations 7ms

*************FAILURES*************
repl.MdocSessionMdocAppExpectationsSuite
- Simple expectations (failure) 2ms
  assertion failed (expectations.md:194)

  expect(clue(test(z)) > z + 6)

  Clues {
    test(z): Int = 20
  }

  expectations.md:194
      expect(clue(test(z)) > z + 6)
            ^
- And/Or composition (failure) 9ms
 [0] assertion failed (expectations.md:202)
 [0] 
 [0] expect(2 < clue(1))
 [0] 
 [0] Clues {
 [0]   1: Int = 1
 [0] }
 [0] 
 [0] expectations.md:202
 [0]     (expect(1 != clue(2)) and expect(2 < clue(1))) or expect(2 > clue(3))
 [0]                                     ^

 [1] assertion failed (expectations.md:202)
 [1] 
 [1] expect(2 > clue(3))
 [1] 
 [1] Clues {
 [1]   3: Int = 3
 [1] }
 [1] 
 [1] expectations.md:202
 [1]     (expect(1 != clue(2)) and expect(2 < clue(1))) or expect(2 > clue(3))
 [1]                                                             ^
- Varargs composition (failure) 8ms
  assertion failed (expectations.md:212)

  clue(2 + 2) > 5

  Clues {
    2 + 2: Int = 4
  }

  expectations.md:212
      expect.all(1 + 1 < 3, clue(2 + 2) > 5, 4 * 2 < 9)
                ^
- Working with collections (failure 1) 5ms
  Values not equal: (expectations.md:223)

  => Diff (- obtained, + expected)
  -hello
  +world

  expectations.md:223
      forEach(Vector("hello", "world"))(msg => expect.eql(msg, "hello"))
                                                         ^
- Working with collections (failure 2) 6ms
  assertion failed (expectations.md:227)

  expect(clue(i) > 50)

  Clues {
    i: Int = 39
  }

  expectations.md:227
      exists(Option(39))(i => expect(clue(i) > 50))
                                    ^
- Strict equality (failure 1) 0ms
  Values not equal: (expectations.md:242)

  => Diff (- obtained, + expected)
  -world
  +hello

  expectations.md:242
      expect.eql("hello", "world")
                ^
- Strict equality (failure 2) 2ms
  Values not equal: (expectations.md:246)

  => Diff (- obtained, + expected)
  -List(1, 19, 3)
  +List(1, 2, 3)

  expectations.md:246
      expect.eql(List(1, 2, 3), List(1, 19, 3))
                ^
- Strict equality (failure 3) 1ms
  Values not equal: (expectations.md:250)

  => Diff (- obtained, + expected)
  -Test(50.0)
  +Test(25.0)

  expectations.md:250
      expect.eql(Test(25.0), Test(50.0))
                ^
- Relaxed equality comparison (failure) 2ms
  Values not equal: (expectations.md:270)

  => Diff (- obtained, + expected)
  -Hello to 50.0
  +Hello to 25.0

  expectations.md:270
      expect.same(new Hello(25.0), new Hello(50.0))
                 ^
- Non macro-based expectations 0ms
  Condition failed (expectations.md:275)

  expectations.md:275
      if (condition) success else failure("Condition failed")
                                         ^
- Failing fast expectations 7ms
  assertion failed (expectations.md:281)

  expect(clue(h).isEmpty)

  Clues {
    h: String = hello
  }

  expectations.md:281
        _ <- expect(clue(h).isEmpty).failFast
                   ^

Total 17, Failed 11, Passed 6, Ignored 0