Developer Guide¶
Architecture¶
Solium is organized as a module that exposes an API for any javascript application to use. The user would supply a source code string along with a configuration object that determines what exactly solium does with the code.
Solium
refers to an engine (a middleman) that accepts user input (source code & configuration) from one side and rule implementations from another.
A rule implementation can refer to any piece of code that operates on the given solidity code’s Abstract Syntax Tree, points out flaws and suggests fixes.
In a sense, Solium is a generic engine that operates on any given solidity code. The linter itself is a special use case of this engine where the “analyzer” refers to a set of rule implementations that tell whether something in the code looks right or wrong. Because you can write plugins and get access to the complete solidity code, its AST and a solium-exposed set of utility functions to operate on the AST, you can build anything on top of solium that you can imagine!
Solium has a set of core rules for the purpose of linting code.
The frontend of the app is a CLI that a user uses to interact with the Solium module to get things done. The module exposes 2 main functions for usage: lint()
and lintAndFix()
.
Architecture will be explained in more detail in future.
Installation & Setting up the Development Enviroment¶
Make sure you have Node.js and NPM installed on your system.
Install Solium v1
as a local module using npm install --save solium
.
You can now use Solium like:
const Solium = require('solium'),
sourceCode = 'contract fOO_bar { string hola = \'hello\'; }';
const errors = Solium.lint(sourceCode, {
"extends": "solium:recommended",
"plugins": ["security"],
"rules": {
"quotes": ["error", "double"],
"double-quotes": [2], // returns a rule deprecation warning
"pragma-on-top": 1
},
"options": { "returnInternalIssues": true }
});
errors.forEach(console.log);
- Source Code can be either a string or a buffer object
lint()
takes in the source code, followed by the soliumrc configuration object.returnInternalIssues
option tells solium to return internal issues (like rule deprecation) in addition to the lint issues. If this option isfalse
, Solium only returns lint issues. It is recommended that you set it totrue
, otherwise you’re missing out on a lot ;-)lint()
returns an array of error objects. The function’s output looks something like:
[
{
type: 'warning',
message: '[DEPRECATED] Rule "double-quotes" is deprecated. Please use "quotes" instead.',
internal: true,
line: -1,
column: -1
},
{
ruleName: 'quotes',
type: 'error',
node: { type: 'Literal', value: 'hello', start: 79, end: 86 },
message: '\'hello\': String Literals must be quoted with double quotes only.',
line: 7,
column: 15,
fix: { range: [Array], text: '"hello"' }
}
]
- You can use the
lintAndFix()
function as demonstrated in the following example:
const Solium = require('solium'),
sourceCode = 'contract fOO_bar { string hola = \'hello\'; }';
const result = Solium.lintAndFix(sourceCode, {
"extends": "solium:recommended",
"plugins": ["security"],
"rules": {
"quotes": ["error", "double"],
"double-quotes": [2], // returns a rule deprecation warning
"pragma-on-top": 1
},
"options": { "returnInternalIssues": true }
});
console.log(result);
The output of lintAndFix()
look like:
{
originalSourceCode: 'pragma solidity ^0.4.0;\n\n\nimport "./hello.sol";\n\ncontract Foo {\n\tstring hola = \'hello\';\n}\n',
fixesApplied:[
{
ruleName: 'quotes',
type: 'error',
node: [Object],
message: '\'hello\': String Literals must be quoted with double quotes only.',
line: 7,
column: 15,
fix: [Object]
}
],
fixedSourceCode: 'pragma solidity ^0.4.0;\n\n\nimport "./hello.sol";\n\ncontract Foo {\n\tstring hola = "hello";\n}\n',
errorMessages: [
{
type: 'warning',
message: '[DEPRECATED] Rule "double-quotes" is deprecated. Please use "quotes" instead.',
internal: true,
line: -1,
column: -1 },
{ ruleName: 'double-quotes',
type: 'warning',
node: [Object],
message: '\'hello\': String Literals must be quoted with "double quotes" only.',
line: 7,
column: 15
}
]
}
Note
The input supplied to lint()
and lintAndFix()
is the same. Its the output format that differs.
To work with Solium:
- clone the repository to your local machine using, for eg,
git clone git@github.com:duaraghav8/Solium.git
. - Move into its directory using
cd Solium
. - Install all dependencies and dev dependencies using
npm install --dev
. - To ensure that everything works fine, run
npm test
. If you’ve cloned themaster
branch, there should be no test failures. If there are, please raise an issue or start a chat on our Gitter channel.
Writing a Core Rule¶
To write a core rule for Solium, please start by raising an issue on github describing your proposal. You can check out some of the rules in the roadmap in our Rules Wishlist.
Note
You are allowed (even encouraged) to write any code you wish to contribute in ES6
.
Say you want to develop a new rule foo-bar
. Here’s how you’d go about it:
Creating a core rule¶
Create a file foo-bar.js
inside lib/rules. This is the main implementation of your rule. Use the below template to implement your core rule:
module.exports = {
meta: {
docs: {
recommended: true,
type: 'warning', // 'warning' | 'error' | 'off'
description: 'This is my foobar rule'
},
schema: [],
fixable: 'code'
},
create(context) {
function lintIfStatement(emitted) {
const { node } = emitted;
if (emitted.exit) {
return;
}
context.report({
node,
fix(fixer) {
// fix logic
},
message: 'Oh snap! A lint error:('
});
}
return {
IfStatement: lintIfStatement
};
}
};
Your rule should expose an object that contains 2 attributes - meta
object which describes the rule and create()
function that actually lints over the given solidity code.
meta
- Contains
docs
object used to describe the rule. - The
schema
object is used to describe the schema of options the user can pass to this rule via soliumrc config (see AJV). This ensures that a valid set of options are passed to your rule. You can see the schema of quotes rule to understand how to write the schema for your rule. - The
fixable
attribute can have value as eithercode
orwhitespace
. Set this attribute if your rule also contains fixes for the issues you report. Usewhitespace
if your rule only add/removes whitespace from the code. Else usecode
. - When a rule needs to be deprecated, we can add
deprecated: true
inside meta. We can addreplacedBy: ["RULE NAME"]
inside meta.docs if this rule is to be replaced by a new rule (see deprecated example).
Note
replacedBy
doesn’t force the linter to apply the new rule. Instead, it only throws a warning to the user, notifying them that they’re using a deprecated rule and should consider moving to the new rule(s) specified inside replacedBy
array. Try adding double-quotes: "error"
inside rules
inside your .soliumrc.json
and running the linter.
create()
This function is responsible for actual processing of the contract code, determining whether something is wrong or not, reporting an issue and suggesting fixes.
create() must return an object whose Key is an AST node type, and value is the function to execute on that node (i.e., the handler function). So, for example, IfStatement
is the type of the AST node representing an if
clause and block in solidity.
Note
To know which node type you need to capture, install solparse, parse some sample code into AST, then examine the particular node of interest for its type
field. Specify that type as your return object key. You can see any rule implementation to understand what create()’s return object looks like.
The create() function receives a context
object, which allows you to access the solidity code to be linted and many other things to help your rule work its magic.
context.options
-undefined
if user doesn’t supply any options to your rule through soliumrc. An Array of options otherwise. Solium ensures that the options passed inside the array are fully compliant with theschema
you define for each of them inmeta
. So if a user specifiesfoo-bar: ['error', 'hello', 110, {a: [99]}]
, thenfoo-bar
rule’scontext.options
contains the array['hello', 110, {a: [99]}]
(all but the first item, because the first is the severity of the rule). See options example.context.getSourceCode()
- returns a SourceCode object that gives you access to the solidity code and several functions to operate on it and AST nodes.
The functions exposed by SourceCode object are as follows:
getText (node)
- get source code for the specified node. If no arguments given, it returns the complete source codegetTextOnLine (lineNumber)
- get the complete text on the specified line number (lineNumber is an Integer)getLine (node)
- get the line number on which the specified node’s code startsgetEndingLine (node)
- get the line number on which the specified node’s code endsgetColumn (node)
- get column no. of the first character of the specified node’s codegetEndingColumn (node)
- get column no. of the last character of the specified node’s codegetParent (node)
- get the parent node of the specified nodegetNextChar (node)
- get 1 character after the code of specified nodegetPrevChar (node)
- get 1 character before the code of specified nodegetNextChars (node, charCount)
- get charCount no. of characters after the code of specified nodegetPrevChars (node, charCount)
- get charCount no. of characters before the code of specified nodeisASTNode (arg)
- Returns true if the given argument is a valid (Spider-Monkey compliant) AST NodegetStringBetweenNodes (prevNode, nextNode)
- get the complete code between 2 specified nodes. (The code ranges from prevNode.end (inclusive) to nextNode.start (exclusive) )getLines ()
- get the source code split into linesgetComments()
- get the list of AST nodes representing comments in the code. CallgetSourceCode()
inside your handler function if you wish to use this method.
Note
The recommended way to use the getSourceCode()
method is inside the handler function in which you will be calling the functions the SourceCode object provides. If you call getSourceCode()
inside the main create()
function, some functions will return empty results because the data hasn’t been populated yet. This is by design. If you see some rule implementations calling the function outside of their handler functions, it means that the SourceCode object functions they use are unaffected by whether you call them inside or outside the handler functions.
context.report()
- Lastly, the context object provides you with a clean interface to report lint issues:
context.report({
node, // the AST node retrieved through emitted.node (see below)
fix(fixer) { // [OPTIONAL]
if (wantToApplyFix) {
return [fixer.replaceText(node, "hello world!!")];
}
return null;
},
message: 'Lint issue raised yayy!',
location: { // [OPTIONAL]
line: 9, // [OPTIONAL]
column: 20 // [OPTIONAL]
}
});
See report with fix example and report with location example.
Note
If you’re supplying the fix()
function, make sure you specify the fixable
attribute in meta
.
Your fix()
function will receive a fixer
object that exposes several functions so you can tell Solium how to fix the raised lint issue. Every fixer function you call returns a fixer packet. Solium understands how to work with this packet. Your fix function must return either a single fixer packet, an array of fixer packets or null
.
Note
Returning a null
results in the particlar fix function being ignored. This is convenient when, under certain conditions, you don’t want to apply any fixes.
This means that fix(fixer) { return null; }
is equivalent to not supplying a fix()
function in the error object at all.
See the context.report()
example above.
Warning
Multiple fixer packets inside the array must not overlap, else Solium throws an error. For eg- the first packet tries to remove the first 10 characters from the solidity code, whereas another packet tries to replace them by, say, “hello world”. This results in an overlap and hence the complete fix is not valid. However, if the replacement begins at the 11th character, then there is no conflict and so your fix is valid!
Below is the list of functions exposed by the fixer
object:
insertTextAfter (node, text)
- inserts text after the given nodeinsertTextAfterRange (range, text)
- inserts text after the given rangeinsertTextBefore(node, text)
- inserts text before the given nodeinsertTextBeforeRange(range, text)
- inserts text before the given rangeremove (node)
- removes the given noderemoveRange(range)
- removes text in the given rangereplaceText(node, text)
- replaces the text in the given nodereplaceTextRange(range, text)
- replaces the text in the given rangeinsertTextAt(index, text)
- inserts text at the given position in the source code
Where range
is an array of 2 unsigned integers, like [12, 19]
, node
is a valid AST node retrieved from emitted.node
(see below), text
is a valid string and index
is an unsigned integer like 69
.
emitted
As mentioned earlier, create()
should return an object. The function specified as the value for a key is responsible for operating over that AST node, so it gets passed an emitted
object. This object’s properties are as follows:
emitted.exit
- Solium passes an AST node to a rule twice - once when it enters the node during its Depth-first traversal and second when its leaving it. exit property, if true, means Solium is leaving the node. So if you only want your rule to execute once over a node, you can specifyif(emitted.exit) { return; }
.
Note
A common use case for exit
is when you want your rule to access the whole contract’s AST Node (type Program
) at the end, ie, when all other rules are done reporting their rules. Then you could specify if(!emitted.exit) { return; }
.
emitted.node
- is the AST Node object of type specified as the key in your return object. So if, for eg, your create() returns{ ForStatement: inspectForLoop }
, then you can access the AST Node representing thefor
loop in solidity like:
create(context) {
function inspectForLoop(emitted) {
const {node} = emitted;
console.log (node.type); // prints "ForStatement" and the node has appropriate properties of 'for' statement
}
return { ForStatement: inspectForLoop };
}
You now have all the required knowledge to develop your core rule lib/rules/foo-bar.js
. Its now time to write tests.
Testing your Core rule¶
- Inside the
test/lib/rules
, creating a new directoryfoo-bar
and a file inside this directoryfoo-bar.js
(see test examples). - Now paste the below template in
test/lib/rules/foo-bar/foo-bar.js
:
/**
* @fileoverview Description of the rule
* @author YOUR NAME <your@email>
*/
'use strict';
const Solium = require('../../../../lib/solium'),
wrappers = require('../../../utils/wrappers');
const { toContract, toFunction } = wrappers;
// Solium should only lint using your rule so only issues flagged by your rule are reported
// so you can easily test it. Replace foo-bar with your rule name.
const config = {
"rules": {
"foo-bar": "error" // alternatively - ["error" OR "warning", options according to meta.schema of rule]
}
};
describe('[RULE] foo-bar: Rejections', () => {
it('should reject some stuff', done => {
const code = 'contract Blah { function bleh() {} }',
errors = Solium.lint(code, config);
// YOUR TESTS GO HERE. For eg:
errors.should.be.size(2); // If you're expecting your rule to flag 2 lint issues on the given code.
Solium.reset();
done();
});
});
describe('[RULE] foo-bar: Acceptances', () => {
it('should accept some stuff', done => {
// YOUR LINTING & TESTS GO HERE. For eg:
Solium.reset();
done();
});
});
You’re now ready to write your tests (see shouldjs documentation).
After writing your tests, add an entry for your rule foo-bar
in solium json.
You also need to add your rule’s entry to the List of Style Rules
section in User Guide.
Finally, add an entry for your rule in solium all and solium recommended rulesets: foo-bar: <SEVERITY>
where severity should be how your rule should be treated by default (as an error or warning). Severity should be same as what you specified in your rule’s meta.docs.type
.
Now run npm run lint
to let eslint work its magic. Resolve any lint issues you might see in your rule & test files.
Run npm test
and resolve any failures.
Once everything passes and there are no lint issues, you’re ready to make a Pull Request :D
Note
ESLint allows us to disable linting on specific pieces of code. This should only be used after a brief discussion about why it’s suitable.
Note
Running npm test
also prints coverage stats at the bottom of the CLI output. It creates the coverage
directory whose index.html
can be opened in any browser to view the same. Write enough tests to keep the coverage for the rule above 90%
.
Developing a Sharable Config¶
The purpose of a sharable config is for an organisation to just pick up a solidity style spec to work with and focus on the coding part instead of getting into a tabs vs. spaces debate. You install the SC and specify its name without prefix as value of the extends
key in your soliumrc config. Something like:
{
"extends": "foobar"
}
(See full documentation in User Guide)
Sharable configs are distributed as modules via NPM. You are encouraged to include solium
, solidity
and soliumconfig
tags in your package.json
. Say, you want to call your config foobar
. Then your module’s name must be solium-config-foobar
. The prefix is mandatory for solium to recognise the module as a sharable config.
Note
For reasons discussed on our blog, we have reserved a few NPM solium config module names. If you find your organisation’s name in the list in the blog, please follow the instructions at the bottom of the blog to claim your module.
Start by creating a directory to contain your module
mkdir solium-config-foobar
cd solium-config-foobar
npm init
Fill in the appropriate details and don’t forget to add the tags mentioned above!- Create your
index.js
file (or whichever you specified as your entry point file). This file must expose an object like below:
module.exports = {
rules: {
quotes: ["error", "double"],
indentation: ["warning", 4],
"pragma-on-top": 1,
...
}
};
- Specify the
peerDependencies
attribute in yourpackage.json
like:
{
...
"peerDependencies": {
"solium": "^1.0.0"
}
}
Read about Peer Dependencies on NPM. You’re now ready to test your config.
Testing your Sharable Config¶
Solium internally simply require()
s the config you extend from in your soliumrc. So as long as require() can resolve the name solium-config-foobar
, it doesn’t care where the config is installed.
The simplest way to test is to first link your config and make it globally available. Traverse to your config directory and run npm link
. You can verify that your config is globally available by going to any random directory, opening a node REPL and running require('solium-config-foobar')
.
Next, go to your dapp directory that contains the .soliumrc.json
file. Open this file and set "extends": "foobar"
(only the config name, not the prefix). You can omit the entire rules
object.
Now run solium -d contracts/
. The linter should behave according to the severities & rule options provided by you.
That’s it! You’re now ready to npm publish
your Sharable Config.
Note
It is a good practice to specify all the rules in your sharable config. This ensures that you decided how each rule is to be treated and that you didn’t forget about any of them. If you wish to turn a rule off, simply specify its value as off
or 0
. See list of all rules on User Guide. See example configuration solium all ruleset.
Note
It is good practice to turn off all the deprecated rules. See the Rule List in User Guide to know which rules are now deprecated.
Developing a Plugin¶
Plugins allow third party developers to write rule implementations that work with solium and re-distribute them for use.
Plugins too are distributed via NPM, have the prefix solium-plugin-
and should, as a best practice, have the tags solium
, solidity
and soliumplugin
.
As an example, you can check out Solium’s official Security Plugin.
Note
For reasons discussed on our blog, we have reserved a few NPM solium plugin module names. If you find your organisation’s name in the list in the blog, please follow the instructions at the bottom of the blog to claim your module.
Start by creating a directory to contain your plugin (lets call the plugin baz
)
mkdir solium-plugin-baz
cd solium-plugin-baz
npm init
Fill in the appropriate details and don’t forget to add the tags mentioned above- Specify the
peerDependencies
attribute in yourpackage.json
like:
{
...
"peerDependencies": {
"solium": "^1.0.0"
}
}
Read about Peer Dependencies on NPM.
- Create your
index.js
file (or whichever you specified as your entry point file). This file must expose an object like below:
module.exports = {
meta: {
description: 'Plugin description'
},
rules: {
foo: {
meta: {
docs: {
recommended: true,
type: 'warning',
description: 'Rule description'
},
schema: []
},
create(context) {
function inspectProgram (emitted) {
if (emitted.exit) {
return;
}
context.report ({
node: emitted.node,
message: 'The rule baz/foo reported an error successfully.'
});
}
return {
Program: inspectProgram
};
}
}
}
};
Note
In the above example, you can set the type
property to off
. The effect of this is that the rule exists in your plugin but is disabled by default. This feature can be used when you require that a user only purposely enable the rule (probaby because it may not be desirable for general audience).
Notice that every rule you define inside the rules
object has the exact same schema as the core rule described above. So if you know how to implement a core rule, you need not learn anything new to implement a plugin rule.
Testing your Plugin¶
Inside your main plugin directory itself:
- Install solium v1 as a dev dependency using
npm install --save-dev solium
. - Run
npm install --save-dev mocha chai should
to install the devDependencies for testing purposes. - In your
package.json
, add the following key:
"scripts": {
"test": "mocha --require should --reporter spec --recursive"
},
- Run
npm link
to make this plugin globally available. (You can confirm that it worked by going to any random directory in your system, firing up Nodejs REPL and runrequire('solium-plugin-baz')
). - Write your tests inside the
test/
directory following the below pattern:
const Solium = require ('solium');
/**
* If you require any other modules like lodash, install them.
* If the module is only being used in your tests, then it should go in your dev dependencies.
* If being used by any of your rules, then it must go into dependencies.
*/
const config = {
plugins: ['baz'],
rules: {
'baz/foo': 'warning'
},
// This returns internal warnings, like deprecation notices
options: {
returnInternalIssues: true
}
};
describe ('Rule foo: Acceptances', () => {
it ('should accept some stuff and reject other stuff', done => {
const code = 'contract BlueBerry { function foo () {} }';
const errors = Solium.lint (code, config);
// If your rules also contain fix()es you'd like to test, use:
// var errors = Solium.lintAndFix (code, config);
console.log ('Errors:\n', errors);
// Now you can test the error objects returned by Solium.
// Each item in errors array represents a lint error produced by the plugin's rules foo & bar
errors.should.be.Array ();
errors.should.have.size (2);
// Add further tests to examine the error objects
// Once your tests have finished, call below functions to safely exit
Solium.reset ();
done ();
});
});
Notice that the schema of plugin rule tests is the same as that of core rule tests.
- Now run the tests using
npm test
and resolve any failures that occur. - As another (optional) test, you can also go to your DApp directory and add your plugin’s entry in
.soliumrc.json
to see if its working properly:
{
"plugins": ["baz"],
"rules": {
"baz/foo": "error"
}
}
And run the linter.
Once all tests pass, you can remove the global link of your plugin using npm unlink
inside your plugin directory and then npm publish
it!
See a sample plugin for solium.
Building this documentation¶
This documentation is built with Sphinx and written in RST.
- To make changes in it, start by cloning Solium to your workstation with
git clone
. cd
into thedocs/
directory. This dir is responsible for containing all rst files, sphinx confguration and builds.- Make sure you have all Sphinx dependencies installed (see getting started with readthedocs).
Note
This documentation builds successfully with Sphinx v1.5
but fails with v1.6
. Although we haven’t yet fully investigated whether its a problem with our docs or Sphinx, we recommend you to install v1.5
in order to see the changes you’ve made.
- Make the changes to the docs as you see fit, then run
make html
while still insidedocs/
. If there were no RST errors, the docs should build successfully. - Open up
docs/_build/html/index.html
in your favourite browser to see the changed. - Once you’re satisfied, you can commit the changes you made in the RST docs and send a PR.