Martin.bravenboer.name

Reusable and Adaptable Strategies for Generative Programming Position paper for the GPCE’04 Software Transformation Systems Workshop Institute of Information and Computing Sciences, Universiteit Utrecht, P.O. Box 80089 3508 TB Utrecht, The Netherlands {martin,visser}@cs.uu.nl Generative programming aims at increasing programmer productivity by automating pro-gramming tasks using some form of automatic program generation or transformation, such ascode generation from a domain-specific language, aspect weaving, optimization, or specializationof a generic program to a particular context. Key for achieving this aim is the construction oftools that implement the automating transformations. If generative programming is to becomea staple ingredient of the software engineering process, the construction of generative tools itselfshould be automated as much as possible. This requires an infrastructure with support for thecommon tasks in the construction of transformation systems. In the Stratego/XT project wehave built a generic infrastructure for program transformation. In this position paper wegive an outline of this infrastructure and indicate where we think the challenges for research anddevelopment of program transformations systems lie in the coming years.
Stratego/XT and its applications are organized in five layers (see diagram). (1) At the bottom layer is the substrate for a transformation system, that is the data representation and exchangeformat, for which we use the Annotated Term Format (ATerm) as basis, and XML where nec-essary to communicate with external tools. This substrate allows transformation systems to berigorously componentized. (2) The foundation of any transformation system are syntax defi-nition and parsing, pretty-printing, program transformation, and tool composition. The syntaxdefinition formalism SDF provides modular syntax definition and parsing, supporting easy com-bination of languages. The pretty-printing package GPP supports rendering structured programrepresentations as program text. The program transformation language Stratego supports conciseimplementation of program transformations by means of rewrite rules and programmable strate-gies for control of their application. Finally, the XTC library supports composition of simpletransformation tools into complex ones. These are all generic facilities that are needed in anytransformation for any language. (3) In the middle is a library of transformations and transfor-mation utilities that are not specific for a language, but not usable in all transformations either.
The Stratego Standard Library (SSL) provides a host of generic rewriting strategies, and the XTtoolset provides utilities for generating parts of a transformation system. (4) Near the top are spe-cializations of the generic infrastructure to specific object languages. Such a language specificenvironment consists of a syntax definition for a language along with utilities such as semantic analysis, variable renaming, and module flattening. (5) Finally, at the top are the actual gen-erative tools such as compilers, language extensions, static analysis tools, and aspect weavers.
These tools are implemented as compositions of tools from the lower layers extended with one ormore components implementing the specific transformation under consideration.
Where are we now in this development? The basic infrastructure (bottom two/three) layers is well established, readily available, and deployed in several projects. Although these componentsare gradually improved and extended, they form a reasonably stable development platform. Thenext frontier is the expansion of the infrastructure to generative applications. The tools in thetop-level layer are the ones that matter since they are to be used by application programmers.
Although we have also built a range of prototype language specific environments and generatorsto experiment with and validate our infrastructure, these do not yet form a product line bythemselves. The main challenges to larger scale deployment are availability of standard languagespecific components, lack of documented reusable strategies at all levels of granularity, and anapproach to make transformation strategies adaptable.
Although the languages and tools provided by Stratego/XT make the construction of trans- formation systems a lot easier than doing it from scratch using a general purpose programminglanguage, it can still be a lot of work. Before a particular generative tool can be created, thehurdle of creating a language-specific environment must be taken. Therefore, the routine produc-tion of generative tools requires a library of standard language specific components, that is,front-ends for many standard and non-standard languages. The expectation that such front-endswill appear as spin-offs from production of specific generative tools is unjustified. The hardnessand tediousness of this problem seems to be caused by the sheer complexity and irregularity ofprogramming languages. Nonetheless, investigation of a higher-level and more declarative solutionto the implementation of front-ends for real programming languages would we worthwhile.
Another challenge for transformation systems is to raise the level of reuse by providing reusable strategies at all levels of granularity, from micro-level transformations to macro-level tool com-positions upto process strategies. As with all reuse, a sensible documentation and indexing ofavailable solutions is key to benefitting from the available components. This is clearly a problemfor new and even for more experienced users of Stratego/XT. With some 170 executable compo-nents and over 1200 strategy definitions in the library, it can be hard to locate the right ones fora specific transformation. While the available components form a very expressive programmingenvironment, supporting the easy composition of all kinds of transformation systems, it requiresknowledge of the available components and the ways to compose them. The cause of this problemis that the available components live at a lower level of abstraction than the problem being solved.
To improve the productivity of meta-programmers we need higher-level semantic strategies thatcapture all aspects of some type of transformation, from the composition of tools to the traversalsused in the actual transformation. Thus, one would expect reusable strategies for compilation,code generation, and aspect weaving, for example. For example, currently, the production of acode generator involves finding components for parsing, pre-processing, and pretty-printing andwriting the actual generation strategy and rules. Often the code for such a generator will lookvery much like other generators, i.e. follow the same design pattern. A reusable code genera-tion strategy should provide a standard composition and a pluggable traversal strategy that onlyneeds to be instantiated with the name of the input and output languages and the generationrules. An ontology for such semantic strategies can then provide access to the components of theinfrastructure starting at the right level of abstraction.
It is not to be expected that such reusable strategies will be right for every possible situation.
Therefore, these strategies should be open and adaptable. In the first place the strategiesshould be adaptable to the specific languages being transformed. In the second place strategiesshould be adaptable to the specific program that is being transformed or generated. Regardingthe first point, languages differ in large and small ways from the standard model. A reusablestrategy based on some general model of data-flow, say, needs to be adapted to the specific data-flow rules of the language under consideration. Regarding the second point: It is not uncommonthat the programs produced by generators, especially those that generate high-level programs, arefurther edited by application programmers in order to fit the generated code in their application.
Generators that produce code templates even assume this mode of work. Of course, this producesa maintenance nightmare; when the source of generation changes, all modified files need to bemerged with the newly generated ones, undoing all benefits of generation. Thus, it should bepossible to adapt the product of generation without physically modifying it, either by means ofinput to the generator, by generating code that can be adapted (e.g. through subclassing), or byapplying further transformations to the generated code (e.g. using aspects).

Source: http://martin.bravenboer.name/docs/BV04-STS.pdf