Skip to content

renovate: add test that should fail#137

Open
nadiamoe wants to merge 2 commits intomainfrom
renovate-tests-refactor
Open

renovate: add test that should fail#137
nadiamoe wants to merge 2 commits intomainfrom
renovate-tests-refactor

Conversation

@nadiamoe
Copy link
Copy Markdown
Contributor

@nadiamoe nadiamoe commented Feb 2, 2023

Meant as an example rather than the actual refactor.

Renovate tests are currently missing some cases that should making them fail.

Moreover, when they fail, the output is very unreadable.

For example, consider this change:

diff --git a/src/changelog/sources/renovate/source_test.go b/src/changelog/sources/renovate/source_test.go
index 0471894..f4bf520 100644
--- a/src/changelog/sources/renovate/source_test.go
+++ b/src/changelog/sources/renovate/source_test.go
@@ -85,7 +85,7 @@ func TestSource_Source(t *testing.T) {
 * Bum chart's version
 * fix typo in the common library`},
 				{Message: "chore(deps): update helm release common-library-2 to v0.0.4"},
-				{Message: "chore(deps): update helm release common-library-3/library to v1.2.3"},
+				{Message: "bogus commit"},
 				{Message: "chore(deps): update helm release common-library-4 to v10.20.30"},
 				{Message: "chore(deps): update helm release common-library-5 to v1.1.2-prerelease+meta"},
 				{Message: "chore(deps): update helm release common-library-6 to v1.1.2+meta"},

The output of the test would be gigantic, because all expectations are offset by one:

Gigantic code block 
?   	github.com/newrelic/release-toolkit	[no test files]
?   	github.com/newrelic/release-toolkit/src/app	[no test files]
ok  	github.com/newrelic/release-toolkit/src/app/common	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/generate	(cached)
?   	github.com/newrelic/release-toolkit/src/app/gha	[no test files]
ok  	github.com/newrelic/release-toolkit/src/app/isheld	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/link	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/nextversion	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/render	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/update	(cached)
ok  	github.com/newrelic/release-toolkit/src/app/validate	(cached)
ok  	github.com/newrelic/release-toolkit/src/bump	(cached)
ok  	github.com/newrelic/release-toolkit/src/bumper	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/linker	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/linker/mapper	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/renderer	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/sources/dependabot	(cached)
ok  	github.com/newrelic/release-toolkit/src/changelog/sources/markdown	(cached)
?   	github.com/newrelic/release-toolkit/src/changelog/sources/markdown/headingdoc	[no test files]
ok  	github.com/newrelic/release-toolkit/src/changelog/sources/markdown/merger	(cached)
time="2023-02-02T12:08:09+01:00" level=warning msg="Renovate could not extract updated version from \"chore(deps): update helm release common-library-3\""
time="2023-02-02T12:08:09+01:00" level=warning msg="Renovate could not extract updated version from \"chore(deps): update integrations (#124)\""
--- FAIL: TestSource_Source (0.00s)
    --- FAIL: TestSource_Source/Matching_commits (0.00s)
        source_test.go:256: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:256
            	Error:      	Not equal: 
            	            	expected: 41
            	            	actual  : 40
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-3/library"
            	            	actual  : "common-library-4"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-3/library
            	            	+common-library-4
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3"
            	            	actual  : "10.20.30"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3
            	            	+10.20.30
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-4"
            	            	actual  : "common-library-5"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-4
            	            	+common-library-5
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "10.20.30"
            	            	actual  : "1.1.2-prerelease+meta"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-10.20.30
            	            	+1.1.2-prerelease+meta
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-5"
            	            	actual  : "common-library-6"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-5
            	            	+common-library-6
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.1.2-prerelease+meta"
            	            	actual  : "1.1.2+meta"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.1.2-prerelease+meta
            	            	+1.1.2+meta
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-6"
            	            	actual  : "common-library-7"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-6
            	            	+common-library-7
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.1.2+meta"
            	            	actual  : "1.1.2+meta-valid"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.1.2+meta
            	            	+1.1.2+meta-valid
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-7"
            	            	actual  : "common-library-8"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-7
            	            	+common-library-8
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.1.2+meta-valid"
            	            	actual  : "1.0.0-alpha"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.1.2+meta-valid
            	            	+1.0.0-alpha
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-8"
            	            	actual  : "common-library-9"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-8
            	            	+common-library-9
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha"
            	            	actual  : "1.0.0-alpha.beta.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha
            	            	+1.0.0-alpha.beta.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-9"
            	            	actual  : "common-library-10"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-9
            	            	+common-library-10
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha.beta.1"
            	            	actual  : "1.0.0-alpha.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha.beta.1
            	            	+1.0.0-alpha.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-10"
            	            	actual  : "common-library-11"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-10
            	            	+common-library-11
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha.1"
            	            	actual  : "1.0.0-alpha0.valid"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha.1
            	            	+1.0.0-alpha0.valid
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-11"
            	            	actual  : "common-library-12"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-11
            	            	+common-library-12
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha0.valid"
            	            	actual  : "1.0.0-alpha.0"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha0.valid
            	            	+1.0.0-alpha.0
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-12"
            	            	actual  : "common-library-13"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-12
            	            	+common-library-13
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha.0"
            	            	actual  : "1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha.0
            	            	+1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-13"
            	            	actual  : "common-library-14"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-13
            	            	+common-library-14
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay"
            	            	actual  : "2.0.0-rc.1+build.123"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha-a.b-c-somethinglong+build.1-aef.1-its-okay
            	            	+2.0.0-rc.1+build.123
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-14"
            	            	actual  : "common-library-15"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-14
            	            	+common-library-15
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.0.0-rc.1+build.123"
            	            	actual  : "1.2.3-SNAPSHOT-123"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.0.0-rc.1+build.123
            	            	+1.2.3-SNAPSHOT-123
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-15"
            	            	actual  : "common-library-16"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-15
            	            	+common-library-16
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3-SNAPSHOT-123"
            	            	actual  : "2.0.0+build.1848"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3-SNAPSHOT-123
            	            	+2.0.0+build.1848
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-16"
            	            	actual  : "common-library-17"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-16
            	            	+common-library-17
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.0.0+build.1848"
            	            	actual  : "2.0.1-alpha.1227"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.0.0+build.1848
            	            	+2.0.1-alpha.1227
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-17"
            	            	actual  : "common-library-18"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-17
            	            	+common-library-18
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.0.1-alpha.1227"
            	            	actual  : "1.0.0-alpha+beta"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.0.1-alpha.1227
            	            	+1.0.0-alpha+beta
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-18"
            	            	actual  : "common-library-19"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-18
            	            	+common-library-19
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0-alpha+beta"
            	            	actual  : "1.2.3----RC-SNAPSHOT.12.9.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0-alpha+beta
            	            	+1.2.3----RC-SNAPSHOT.12.9.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-19"
            	            	actual  : "common-library-20"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-19
            	            	+common-library-20
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3----RC-SNAPSHOT.12.9.1"
            	            	actual  : "1.0.0+0.build.1-rc.10000aaa-kk-0.1"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3----RC-SNAPSHOT.12.9.1
            	            	+1.0.0+0.build.1-rc.10000aaa-kk-0.1
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-20"
            	            	actual  : "common-library-21"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-20
            	            	+common-library-21
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.0.0+0.build.1-rc.10000aaa-kk-0.1"
            	            	actual  : "999999.999999.999999"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.0.0+0.build.1-rc.10000aaa-kk-0.1
            	            	+999999.999999.999999
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "common-library-21"
            	            	actual  : "rust crate i-do-not-exist"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-common-library-21
            	            	+rust crate i-do-not-exist
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "999999.999999.999999"
            	            	actual  : "1.2.3"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-999999.999999.999999
            	            	+1.2.3
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "rust crate i-do-not-exist"
            	            	actual  : "noprefix"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-rust crate i-do-not-exist
            	            	+noprefix
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "noprefix"
            	            	actual  : "name with spaces"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-noprefix
            	            	+name with spaces
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "name with spaces"
            	            	actual  : "fancy-module"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-name with spaces
            	            	+fancy-module
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "fancy-module"
            	            	actual  : "module"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-fancy-module
            	            	+module
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "module"
            	            	actual  : "newrelic/infrastructure-bundle"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-module
            	            	+newrelic/infrastructure-bundle
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "1.2.3"
            	            	actual  : "2.7.6"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-1.2.3
            	            	+2.7.6
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"280", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) "",
            	            	+ PR: (string) (len=3) "280",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "newrelic/infrastructure-bundle"
            	            	actual  : "github.com/google/go-github/v39"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-newrelic/infrastructure-bundle
            	            	+github.com/google/go-github/v39
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "2.7.6"
            	            	actual  : "39.2.0"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-2.7.6
            	            	+39.2.0
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"280", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"123", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) (len=3) "280",
            	            	+ PR: (string) (len=3) "123",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "github.com/google/go-github/v39"
            	            	actual  : "newrelic/infrastructure"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-github.com/google/go-github/v39
            	            	+newrelic/infrastructure
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:260: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260
            	Error:      	Not equal: 
            	            	expected: "39.2.0"
            	            	actual  : "1.20.5"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-39.2.0
            	            	+1.20.5
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"123", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"125", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) (len=3) "123",
            	            	+ PR: (string) (len=3) "125",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "newrelic/infrastructure"
            	            	actual  : "integrations"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-newrelic/infrastructure
            	            	+integrations
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:262: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:262
            	Error:      	Expected nil, but got: &semver.Version{major:1, minor:20, patch:5, pre:"", metadata:"", original:"v1.20.5"}
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:264: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:264
            	Error:      	Not equal: 
            	            	expected: changelog.EntryMeta{Author:"", PR:"125", Commit:""}
            	            	actual  : changelog.EntryMeta{Author:"", PR:"124", Commit:""}
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -2,3 +2,3 @@
            	            	  Author: (string) "",
            	            	- PR: (string) (len=3) "125",
            	            	+ PR: (string) (len=3) "124",
            	            	  Commit: (string) ""
            	Test:       	TestSource_Source/Matching_commits
        source_test.go:258: 
            	Error Trace:	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:258
            	Error:      	Not equal: 
            	            	expected: "integrations"
            	            	actual  : "aquasecurity/trivy-action"
            	            	
            	            	Diff:
            	            	--- Expected
            	            	+++ Actual
            	            	@@ -1 +1 @@
            	            	-integrations
            	            	+aquasecurity/trivy-action
            	Test:       	TestSource_Source/Matching_commits
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x51fd39]

goroutine 9 [running]:
testing.tRunner.func1.2({0x61e940, 0x821f90})
	/usr/lib/go/src/testing/testing.go:1396 +0x24e
testing.tRunner.func1()
	/usr/lib/go/src/testing/testing.go:1399 +0x39f
panic({0x61e940, 0x821f90})
	/usr/lib/go/src/runtime/panic.go:884 +0x212
github.com/Masterminds/semver.(*Version).String(0x0)
	/home/roobre/.go/pkg/mod/github.com/!masterminds/semver@v1.5.0/version.go:113 +0x59
github.com/newrelic/release-toolkit/src/changelog/sources/renovate_test.TestSource_Source.func1(0xc0001d01a0)
	/home/roobre/.go/src/github.com/newrelic/release-toolkit/src/changelog/sources/renovate/source_test.go:260 +0x385
testing.tRunner(0xc0001d01a0, 0xc00002f100)
	/usr/lib/go/src/testing/testing.go:1446 +0x10b
created by testing.(*T).Run
	/usr/lib/go/src/testing/testing.go:1493 +0x35f
FAIL	github.com/newrelic/release-toolkit/src/changelog/sources/renovate	0.006s
ok  	github.com/newrelic/release-toolkit/src/git	(cached)
?   	github.com/newrelic/release-toolkit/src/hack	[no test files]
ok  	github.com/newrelic/release-toolkit/src/version	(cached)
FAIL

This makes very hard to add new tests to the suite, as if the test does not work for one commit, all the others will fail as well. I think we should consider refactoring this test suite so it looks more like Dependabot's, which does not have this problem.

@nadiamoe
Copy link
Copy Markdown
Contributor Author

nadiamoe commented Feb 2, 2023

@alvarocabanas If you could take a look at this it would be awesome ❤️

@alvarocabanas alvarocabanas marked this pull request as ready for review February 22, 2023 08:02
@alvarocabanas alvarocabanas self-requested a review February 22, 2023 08:02
@alvarocabanas alvarocabanas force-pushed the renovate-tests-refactor branch from d290847 to 9559dbd Compare February 22, 2023 08:04
@alvarocabanas alvarocabanas force-pushed the renovate-tests-refactor branch from 9559dbd to 822600c Compare February 22, 2023 08:14
alvarocabanas
alvarocabanas approved these changes Feb 22, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor

@alvarocabanas alvarocabanas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

nadiamoe
nadiamoe commented Mar 13, 2023
View reviewed changes
Copy link
Copy Markdown
Contributor Author

@nadiamoe nadiamoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome @alvarocabanas, thanks a lot!

Left a quick doubt and after this we can merge this :)

nadiamoe
nadiamoe commented Mar 13, 2023
View reviewed changes
Comment thread src/changelog/sources/renovate/source_test.go
}

//nolint:funlen
//nolint:funlen,paralleltest,maintidx
Copy link
Copy Markdown
Contributor Author

@nadiamoe nadiamoe Mar 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we getting rid of paralleltest? If this is unavoidable I'd love to see a short explanation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alvarocabanas alvarocabanas alvarocabanas approved these changes

Assignees

@alvarocabanas alvarocabanas

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nadiamoe @alvarocabanas