Three Years of Phpactor

The first commit in Phpactor (pronounced “factor”) dates from almost three years ago:

commit 3677c5cb58a5b203fb658c8e2498e512cdef555a
Author: dantleech <[email protected]>
Date:   Thu Sep 24 14:08:35 2015 +0200

    Initial

I had no idea about how to create such an ambitious project in a domain in which I knew nothing. But I had been using VIM for around 7 years (?), VIM is a great text editor, but the tooling around refactoring and auto-completion for PHP was sub-optimal, and instead of waiting more years, I decided to write my own tool.

Actually almost all Phpactor development has happened in the past year-and-a-half. The above commit was the first of three attempts to create a code-completion and refactoring tool backend for editors such as VIM. This initial (3 commit) effort was to use an SQLite database to index the classes and functions in a project (an approach that was later dropped). Then followed another few commits four months later, then more 6 months after that. More serious development started in late 2016 but I struggled with the PhpParser, I then found out about the Microsoft Tolerant PHP Parser which was designed exactly for Phpactor’s use case, I also decided to take a more pragmatic approach to the project.

The Phpactor logo

What single thing would deliver the most value to me, what would provide the biggest return-on-investment?

Completion would have been nice, but the one thing that I wanted the most was a way to move classes - this has been an extremely painful thing to do in VIM, requiring not only moving files, but also updating all of the class namespaces, and all the references to those classes.

I decided to concentrate on this single feature instead of trying to solve the much more difficult problem of code completion. I would just do whatever was necessary to make class moving work, in a separate, fully decoupled, stand-alone library. It wouldn’t matter if the code was sub-optimal as it wouldn’t contaminate other areas of the application. With this in mind I restarted the project:

commit 07a8bbb442966854bc6029e7e8490b151366e69a
Author: dantleech <[email protected]>
Date:   Mon Jun 19 14:47:32 2017 +0100

    Restarting the project

From this point on Phpactor has just continued to grow and it slowly aggregated more and more functionality spread out over different repositories. The class moving library has basically remained the same since it’s creation.

Parameter Completion

The origins of Phpactor can be found in some humble VIM plugins I wrote, one which determines the namespace of the current class file and another which generates a PHPUnit test case for the current class. Both of these plugins made use of the composer autoloader to determine class locations. This non-standard use of the composer autoloader is what powers Phpactor’s source-location abilities, making slow indexing processes and caching largely unnecessary.

The current version of Phpactor is something of an epic project. It has many libraries (for example worse reflection, code-transform, code-builder, docblock, class-mover, completion, path-finder, and more). All of the libraries are untagged and unstable, none of them are intended to be consumed by other projects at this point in time and most of them fall short of being outstanding, providing only what is good-enough for the Phpactor project. They are, however decoupled from the main Phpactor application.

Some of Phpactors Components

An Opportunity to Experiment ¶

One of the advantages of personal projects is that you have freedom to experiment with new ways of doing things, while at work this can either be risky, or inappropriate.

In writing this project I wanted to try some of the DDD concepts I discovered after reading Vaughn Vernons Implementing Domain-Driven Design. Implementing a new paradigm is always going to be a trail-and-error experience, and I would do some things differently next time. All the libraries in Phpactor have a directory structure similar to the following:

lib/
    Core/
        ...
    Adapter/
        ...

With all the “clean” uncoupled code in Core (I was going to call this Domain, but didn’t want to presume that I was doing DDD) and the adapter which implement the interfaces in Core and provide a coupling to another library. In an ideal world these adapters would be in separate packages, but the value wouldn’t outweigh the effort in this case (or at least at this moment in time). I also implemented many Value Objects (VOs).

There is an amount of VO duplication between packages, notably for things such as SourceCode and ClassName objects. It might make sense in the future to extract some of the VO objects to a separate packages, but it’s difficult to determine if the meaning is exactly the same (e.g. a ClassName VO in a library which infers class names from filenames has different requirements than a ClassName in the reflection library), however SourceCode is implemented three or four times as basically the same class.

Extract Method

I also learnt to avoid expose value objects at package boundaries as arguments. Exposing them meant that code outside the package would know more about the package than it ought to have done (for example, it imported the class name, or used a specific method of the VO) and therefore increased the difficulty and risk of refactoring the package.

For example:

use Phpactor\\ClassToFile\\Core\\ClassName as ClassToFileClassName;
use Phpactor\\WorseReflection\\Core\\ClassName as ReflectionClassName;

$file = $classToFile->classToFile(ClassToFileClassName::fromString('Foobar\\Barfoo'));
$reflection = $reflector->reflectClass(ReflectionClassName::fromString('Foobar\\Barfoo'));

and without the VO:

$file = $classToFile->classToFile('Foobar\\Barfoo');
$reflection = $reflector->reflectClass('Foobar\\Barfoo');

Internally the package could still use the VO but this detail is hidden from the outside.

Wheel Reinventing ¶

The wheel has been reinvented a few times, notably in the case of WorseReflection (WR) - the backbone of Phpactor. It provides broadly the same functionality as, and was influenced by, BetterReflection (BR) with the addition of type and value flow (required for completion). The justification here is that it would have been impossible to merge the type-flow code in WR into BR, because it was so bad and experimental. But whilst being experimental it was providing actual value to Phpactor. In addition, BR had some performance problems (at the time) which made it specifically unsuitable for real-time completion.

On one hand it is a shame that I didn’t contribute to BetterReflection, but on the other I don’t think Phpactor would have been built if I did. WR is the core domain, and as such it is subservient to the needs of the project and needs to be owned by it.

Another example of wheel-reinventing is the docblock parser. There is already the PHPDoc DocBlock and the great PHPStan PHPDoc Parser. The first project depended on the nikic/php-parser for type resolution (which is arguably not a requirement for a parser). The PHPStan parser was functionally perfect, and I happily tried to replace the Phpactor parser - but unfortunately it was 10x slower than dumb regex parsing, so the otherwise inferior Phpactor package is still relevant. It’s the difference between a 0.25s completion time on a PHPUnit test case, and a 2.5s one.

Finally there is Phpactor’s RPC protocol used to talk to the editor. At the time I was vaguely aware of Langauge Server Protocol (LSP) but didn’t look more into it as it is for a language server. Phpactor is not a server, it’s invoked as a command. In hindsight the RPC protocol of Phpactor can fit inside the LSP and Phpactor could optionally be made into a server (although running as a short-lived process is better in terms of stability) (see pull request). LSP support would allow Phpactor to be used transparently by many more editors.

Import class and Implement Contract

Return on Investment ¶

Phpactor has taken up a huge amount of my spare time over the past year-and-a-half. I enjoy coding and look forward to spending a Saturday morning crafting a new feature in Phpactor, but often morning becomes mid-afternoon, and sometimes intrudes into Sunday.

Personally Phpactor is now an indispensable tool that I use at work every day, and I am most motivated to work on it when I am presented with a particular challenge in my job.

But I do ask myself if it is worth the time given that there are other projects which have grown in Phpactors lifetime (e.g. Php Language Server and Padawan) but these two libraries are mostly (exclusively?) concerned with code-completion, I don’t think there is anything freely available which competes directly with Phpactor.

But still - is it worth me investing all this time when I could be working on other projects? (like my other side-project, Phpbench, which has seen little attention since I started Phpactor) – or – doing things other than programming?

This is a question I ask myself sometimes, and to be honest, all things considered, it probably isn’t worth it. But I am happy that Phpactor turns VIM into viable modern IDE for PHP and it can now handle finding method references for example, and provide some relief to users of other editors working with VIM users:

I was slightly surprised to notice when I paired with a developer using PHPStorm we both look for references to a particular method, both PHPStorm and Phpactor returned the same methods and, when finding class references, Phpactor actually seemed to have out-performed PHPStorm. I do not assert that Phpactor is as accurate or comprehensive as PHPStorm (because it is not), but it does a pretty good job.

Class References

Finally, there is more interaction with other people in the Phpactor project than on my other projects, although it only has ~280 stars on Github (compared to, for example, Phpbench’s ~780) there are many more people contributing and raising issues and creating third-party integrations (such as a plugin for emacs and integrations with completion manages such as ncm and deoplete). Having this feedback is encouraging,

Phpactor is by no means perfect - it will not find all your references, it will not complete everthing that PHPStorm would, some of the refactorings will, sometimes leave you with incorrect code. But for the most part it works really well.

It may be that one day Phpactor will be displaced by an even better solution, which would be fine. But I hope it will continue to grow and that some of the technical debt can be repaid and that one day some of the libraries will be stable and even more useful refactorings and features will be developed.