Hi all!
I’m the lead dev of a very large project started in 2011 in 6G60, I won’t go into specific details here as it’s extremely NSFW and don’t want to break any rules or ruffle any feathers.
Anyway we recently started having huge problems with 6G60 where the compiler was halting in ‘abject failure’ due to how large and complex the code was (we know it was this because we got to several points where deleting any single function would allow the code to compile, and adding any single empty function would cause it to fail).
So we’ve been trying to upgrade to 6M62 in the hopes that this complexity limit is different, or at the very least we can make a bug report to the main website explaining our issue. Thanks to the work of many people including Dannii Willis, there are now enough extensions compatible with 6M62 that this seemed realistic.
So, I began migrating one of the simplest versions of the code over - the ‘no graphics’ version which doesn’t have any of the complex code for rendering graphics with Glimmr and Flexible Windows and which definitely compiled with ease on 6G60. But already we’ve hit a roadblock - after fixing all the issues that the new 6M62 compiler wasn’t happy with, it’s now just halting in failure with a code 11 and a complicated stack trace.
Since this is a huge codebase with over a million words being migrated from 6G60, it’s not possible to just ‘revert the last change’ and be aware of what is causing the problem. Instead we need to somehow hunt down whatever the thing is that’s causing the compiler to break (or more likely, things) and the only way I can think of to do this is to break down the source code bit by bit until the compiler stops breaking and then build it back up bit by bit afterwards. However this will take weeks if not months and during this time we’re going to lose a lot of income which will significantly affect a number of people’s livelihoods if we can’t manage to find the solution faster.
So I’ve come here to ask: does anyone have a better suggestion of how to go about hunting this issue down? Any wild ideas about what sort of thing we should be looking for? Any clever methods to go about cutting spaghetti code into smaller chunks that can still compile? Currently I’m thinking of trying to write a script that will take every variable and object definition, and take every procedure and function definition and just return ‘nothing’, and lose everything else. And then slowly build it back up from there. I think it will take forever though.
For anyone that’s interested enough to listen, here are the other details I can think of that might be relevant:
I’ve noticed that there’s one sort of issue causing a code 11 that’s already known: inform7.com/mantis/view.php?id=1837 but I think if we suffered from a similar mistake, then it wouldn’t have successfully compiled in 6G60.
The compiler creates a Problems.html file that grows to nearly 250MB before it finally halts. Opening that index file in a browser shows a seemingly infinite repeating nesting of reports:
When using 6G60, we had to use some very large limit overrides, for example
Use MAX_NUM_STATIC_STRINGS of 300000.
Removing these from the source code for 6M62 doesn’t change anything about this compiler error. So I guess that suggests that it’s crashing and burning before any checks are being made against these limits.
Stack Trace:
[spoiler]Process failed, stack backtrace:
Exception code 0xc00000fd at PC=0x76897c97
msvcrt!_unlock()
msvcrt!_write()
msvcrt!_flsbuf()
msvcrt!putc()
ni!Streams__putc() at inform7\Chapter 2\Streams.w line 558
ni!Streams__putc() at inform7\Chapter 2\Streams.w line 599
ni!Streams__printf() at inform7\Chapter 2\Streams.w line 445
ni!Problems__Issue__issue_problems_banner() at inform7\Chapter 6\Problems, Level 3.w line 827
ni!HTML__html_outcome_image() at inform7\Chapter 4\HTML Files.w line 229
ni!Problems__show_problem_location() at inform7\Chapter 6\Problems, Level 2.w line 83
ni!Problems__issue_problem_begin() at inform7\Chapter 6\Problems, Level 2.w line 315
ni!Problems__Issue__internal_error_fn() at inform7\Chapter 6\Problems, Level 3.w line 57
ni!Memory__allocate_another_block() at inform7\Chapter 2\Memory.w line 388
ni!Memory__allocate() at inform7\Chapter 2\Memory.w line 540
ni!allocate_parse_node_annotation_array() at inform7\Chapter 2\Memory.w line 835
ni!allocate_parse_node_annotation() at inform7\Chapter 2\Memory.w line 835
ni!ParseTree__pna_new() at inform7\Chapter 13\Parse Tree.w line 760
ni!ParseTree__pn_annotate_pointer() at inform7\Chapter 13\Parse Tree.w line 843
ni!ParseTree__set_problem_falls_under() at inform7\Chapter 13\Parse Tree.w line 912
ni!Problems__visit_for_headings() at inform7\Chapter 6\Problems, Level 2.w line 51
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1555
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1551
ni!Problems__find_headings_at() at inform7\Chapter 6\Problems, Level 2.w line 35
ni!Problems__show_problem_location() at inform7\Chapter 6\Problems, Level 2.w line 90
ni!Problems__issue_problem_begin() at inform7\Chapter 6\Problems, Level 2.w line 315
ni!Problems__Issue__internal_error_fn() at inform7\Chapter 6\Problems, Level 3.w line 57
ni!Memory__allocate_another_block() at inform7\Chapter 2\Memory.w line 388
ni!Memory__allocate() at inform7\Chapter 2\Memory.w line 540
ni!allocate_parse_node_annotation_array() at inform7\Chapter 2\Memory.w line 835
ni!allocate_parse_node_annotation() at inform7\Chapter 2\Memory.w line 835
ni!ParseTree__pna_new() at inform7\Chapter 13\Parse Tree.w line 760
ni!ParseTree__pn_annotate_pointer() at inform7\Chapter 13\Parse Tree.w line 843
ni!ParseTree__set_problem_falls_under() at inform7\Chapter 13\Parse Tree.w line 912
ni!Problems__visit_for_headings() at inform7\Chapter 6\Problems, Level 2.w line 51
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1555
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1551
ni!Problems__find_headings_at() at inform7\Chapter 6\Problems, Level 2.w line 35
ni!Problems__show_problem_location() at inform7\Chapter 6\Problems, Level 2.w line 90
ni!Problems__issue_problem_begin() at inform7\Chapter 6\Problems, Level 2.w line 315
ni!Problems__Issue__internal_error_fn() at inform7\Chapter 6\Problems, Level 3.w line 57
ni!Memory__allocate_another_block() at inform7\Chapter 2\Memory.w line 388
ni!Memory__allocate() at inform7\Chapter 2\Memory.w line 540
ni!allocate_parse_node_annotation_array() at inform7\Chapter 2\Memory.w line 835
ni!allocate_parse_node_annotation() at inform7\Chapter 2\Memory.w line 835
ni!ParseTree__pna_new() at inform7\Chapter 13\Parse Tree.w line 760
ni!ParseTree__pn_annotate_pointer() at inform7\Chapter 13\Parse Tree.w line 843
ni!ParseTree__set_problem_falls_under() at inform7\Chapter 13\Parse Tree.w line 912
ni!Problems__visit_for_headings() at inform7\Chapter 6\Problems, Level 2.w line 51
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1555
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1551
ni!Problems__find_headings_at() at inform7\Chapter 6\Problems, Level 2.w line 35
ni!Problems__show_problem_location() at inform7\Chapter 6\Problems, Level 2.w line 90
ni!Problems__issue_problem_begin() at inform7\Chapter 6\Problems, Level 2.w line 315
ni!Problems__Issue__internal_error_fn() at inform7\Chapter 6\Problems, Level 3.w line 57
ni!Memory__allocate_another_block() at inform7\Chapter 2\Memory.w line 388
ni!Memory__allocate() at inform7\Chapter 2\Memory.w line 540
ni!allocate_parse_node_annotation_array() at inform7\Chapter 2\Memory.w line 835
SNIP - THE ABOVE 18 LINES REPEAT ABOUT 1000 TIMES
ni!ParseTree__pna_new() at inform7\Chapter 13\Parse Tree.w line 760
ni!ParseTree__pn_annotate_pointer() at inform7\Chapter 13\Parse Tree.w line 843
ni!ParseTree__set_problem_falls_under() at inform7\Chapter 13\Parse Tree.w line 912
ni!Problems__visit_for_headings() at inform7\Chapter 6\Problems, Level 2.w line 51
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1555
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_from_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1558
ni!ParseTree__traverse_ppn_nocs() at inform7\Chapter 13\Parse Tree.w line 1551
ni!Problems__find_headings_at() at inform7\Chapter 6\Problems, Level 2.w line 35
ni!Problems__show_problem_location() at inform7\Chapter 6\Problems, Level 2.w line 90
ni!Problems__issue_problem_begin() at inform7\Chapter 6\Problems, Level 2.w line 315
ni!Problems__Issue__internal_error_fn() at inform7\Chapter 6\Problems, Level 3.w line 57
ni!Memory__allocate_another_block() at inform7\Chapter 2\Memory.w line 388
ni!Memory__allocate() at inform7\Chapter 2\Memory.w line 540
ni!allocate_parse_node() at inform7\Chapter 2\Memory.w line 767
ni!ParseTree__new() at inform7\Chapter 13\Parse Tree.w line 667
ni!Invocations__new() at inform7\Chapter 31\Invocations.w line 146
ni!Phrases__Parser__parse_against() at inform7\Chapter 31\Parse Invocations.w line 253
ni!SParser__add_ilist() at inform7\Chapter 16\Conditions and Phrases.w line 323
ni!s_text_substitution_NTMR() at inform7\Chapter 16\Conditions and Phrases.w line 253
ni!Preform__parse_nt_against_word_range() at inform7\Chapter 7\Preform.w line 1738
ni!Preform__parse_nt_against_word_range() at inform7\Chapter 7\Preform.w line 2103
ni!Preform__parse_nt_against_word_range() at inform7\Chapter 7\Preform.w line 2103
ni!SParser__parse_phrase_inner() at inform7\Chapter 16\Architecture of the S-Parser.w line 304
ni!SParser__parse_say_term() at inform7\Chapter 16\Architecture of the S-Parser.w line 297
ni!Routines__Compile__code_line() at inform7\Chapter 31\Compile Phrases.w line 199
ni!Routines__Compile__code_block() at inform7\Chapter 31\Compile Phrases.w line 154
ni!Routines__Compile__code_line() at inform7\Chapter 31\Compile Phrases.w line 367
ni!Routines__Compile__code_block() at inform7\Chapter 31\Compile Phrases.w line 154
ni!Routines__Compile__routine() at inform7\Chapter 31\Compile Phrases.w line 99
ni!Phrases__compile() at inform7\Chapter 28\Phrases.w line 378
ni!Routines__ToPhrases__compilation_coroutine() at inform7\Chapter 28\To Phrases.w line 263
ni!Phrases__Manager__compile_as_needed() at inform7\Chapter 28\Construction Sequence.w line 503
ni!TemplateFiles__interpret() at inform7\Chapter 38\I6 Template Interpreter.w line 789
ni!TemplateFiles__interpret() at inform7\Chapter 38\I6 Template Interpreter.w line 460
ni!Main__core_inform_main() at inform7\Chapter 38\Main Routine.w line 43
ni!main() at inform7\Chapter 38\Main Routine.w line 15
ni!__tmainCRTStartup() at \usr\src\debug\mingw64-i686-runtime-4.0.6-1\crt\crtexe.c line 334
KERNEL32!BaseThreadInitThunk()
ntdll!RtlValidSecurityDescriptor()
ntdll!RtlValidSecurityDescriptor()
Compiler finished with code 11[/spoiler]