Simple postfix macros

[joshtriplett]

This RFC introduces simple postfix macros, of the form expr.ident!(),
to make macro invocations more readable and maintainable in
left-to-right method chains.

In particular, this proposal will make it possible to write chains like
computation().macro!().method().another_macro!(), potentially with ?
interspersed as well; these read conveniently from left to right rather
than alternating between the right and left sides of the expression.

I believe this proposal will allow more in-depth experimentation in the
crate ecosystem with features that would otherwise require compiler
and language changes, such as introducing new postfix control-flow
mechanisms analogous to ?.

Update: I've rewritten the desugaring to use the same autoref mechanism
that closure capture now uses, so that some_struct.field.mac!() works.
I've also updated the specified behavior of stringify! to make postfix
macros like .dbg!() more useful.

Rendered

[nikomatsakis]

I'm torn on this topic, but ultimately feeling favorable.

On the one hand, sometimes I think "why don't we just make foo.bar!() shorthand for bar!(foo, ...)", which I gather is roughly what you describe here. Except this proposal is opt-in, which is good (see below).

On the other hand, I think there is no fundamental reason that macro-expansion can't be interspersed with type-checking, a la Wyvern or (I think?) Scala. In that case, we could make foo.bar! a true type-based macro invocation, and enable some other nifty things (like selecting which macro to use from context, as Wyvern does).

On the gripping hand, that is so far out as to be science fiction, and the need for postfix macros is real today. Plus, if we ever get there — and indeed if we ever want to get there — I suppose that the $self:self notation could be deprecated and some other notation introduced for a type-based macro.

[petrochenkov]

Just adding a new matcher self would probably be enough - $anything: self.
During matching it wouldn't match anything except for a "method receiver" in expr.my_macro!(...), but during expansion it would work as expr.

[est31]

🚲 🏠: I think in order to be consistent with function syntax it should be $self as in ($self, $msg:expr) => ({.

[joshtriplett]

@petrochenkov That's exactly what I intended. I described the :self as a new "designator", because the Rust documentation used that term. Do you mean something different when you describe it as a "matcher"?

@est31 I considered that possibility; however, in addition to the inconsistency of not using a descriptor, that would limit potential future expansion a bit. Today, you can write $self:expr and use $self, without the compiler attaching any special meaning to the use of the name self as a macro argument. So, making $self without a descriptor special seems inconsistent in a problematic way.

[petrochenkov]

@joshtriplett
I see, the RFC never shows an example with $not_self: self, so I thought that $self is mandatory.

[joshtriplett]

@petrochenkov Fixed.

[est31]

@joshtriplett good point. Ultimately I don't care much about the actual syntax.

[petrochenkov]

cc @nrc who hated the idea

[Centril]

Putting aside some of the more technical aspects of the RFC and focusing solely on the motivation...

This is quite a beautiful RFC. I wholeheartedly support some form of postfix macros;
They make macros feel like methods, which is quite lovely.

Considering some other use cases:

• dbg! -- here a postfix macro would allow you to just have expr and then (proviso that the precedence checks out) simply tack on .dbg!() at the end and get expr.dbg!(). To me, this is the optimal light-weight debugging experience you can get in the playground. Of course, when you have something like x + y you can always switch back to dbg!(x + y). That said, for expr.dbg!() to work well per designs in Simpler alternative dbg!() macro #2361 and RFC: Quick dbg!(expr) macro #2173, then stringify! needs to behave differently than specified in the RFC right now.

• throw! -- you can simply write myError.throw!(); This invocation style makes quite a bit of sense; you are taking an object, and then performing some verb on it actively.

Some wilder considerations, which are mostly contra-factual musings of mine...
In a parallel universe where we started out with postfix macros, one might have considered (keeping in mind the precedence issues...):

• expr.return!()
• expr.break!('label)
• expr.break!()

[est31]

Bit 👍 from me. This would allow for expr.unwrap_or!(return) which I wanted to do so often but couldn't (as well as expr.unwrap_or!(continue)).

[joshtriplett]

@Centril, @est31: Those are exactly the kinds of things I had in mind. I was partly inspired by recent discussions around try blocks and throw/fail; having postfix macros allows for a lot more experimentation prior to deciding something needs elevating to a language built-in.

[dtolnay]

I noticed that $self by itself is currently not valid syntax (as of rust-lang/rust#40107).

error: missing fragment specifier
 --> src/main.rs:2:11
  |
2 |     ($self, rest) => {}
  |           ^

As an alternative could you cover the advantages of $self:self over just $self? Is it about future compatibility, consistency with how other fragments are specified, some sort of ambiguous edge case, etc? I think people are generally used to and like the idea of self not needing a type specified in method signatures.

[dtolnay]

I didn't find this in the RFC but would be worth calling out: is the expectation that $self:self must be followed by either , or )?

[durka]

I think I like this idea, especially given the await motivation, but I can't decide about the evaluation issue.

• If the self argument is not pre-evaluated, it's quite surprising for users: reading a method chain and encountering a macro, you have to revise your understanding of the whole expression so far, like a garden path sentence.
• If the self argument is pre-evaluated as proposed, it's surprising and limiting for macro authors: normally, a macro gets everything unevaluated and can do as it wishes. Macros like launch_missiles().just_kidding!(); (or more practically, let's say install_drivers().only_if_cfg!(windows);) can't be written.

I suspect the tension between these two is a big reason we don't have postfix macros yet.

[scottmcm]

Even just .unwrap!() would mean better locations, without needing #2091...

(And the trait for ? would let it work over Result and Option with the same macro expansion.)

[joshtriplett]

@dtolnay

If the self argument is not pre-evaluated, it's quite surprising for users: reading a method chain and encountering a macro, you have to revise your understanding of the whole expression so far, like a garden path sentence.

Thanks, that's the perfect explanation for what I was trying to get at. I'm going to incorporate that into a revision of the RFC.

If the self argument is pre-evaluated as proposed, it's surprising and limiting for macro authors: normally, a macro gets everything unevaluated and can do as it wishes. Macros like launch_missiles().just_kidding!(); (or more practically, let's say install_drivers().only_if_cfg!(windows);) can't be written.

That's why I'm specifically positioning this as "simple postfix macros". This doesn't preclude adding more complex postfix macros in the future, but it provides a solution for many kinds of postfix macros people already want to write.

[durka]

Reposting a comment that got hidden:

It's not just stringify -- it's any macro that expects an unevaluated expression.

macro_rules! is_ident {
    ($i:ident) => { true };
    ($e:expr) => { false }
}

macro_rules! log {
    ($self:self) => {{
        println!("{:?}", is_ident!($self));
        $self
    }}
}

42.log!();

What does this print? It seems quite surprising for it to print true, but impossible for it to print false.

[joshtriplett]

@durka Why couldn't it print false? My inclination would be for expr or tt to match, but no other designator. (And another postfix macro could capture it as another $self:self, if the first macro wrote $self.othermacro!().)

Does that make sense?

[durka]

It makes sense when you put on compiler-colored glasses, knowing that it's expanded to this invisible temporary binding. But normally macros can tell that xyz is an :ident and 42 is a :literal, etc, so it's weird that this ability is lost with postfix macros.

[joshtriplett]

@durka I understand what you mean, and that thought crossed my mind too. On the other hand, it can't match :literal or :ident, because then someone might try to match it that way and use the result in a context that requires a literal or ident.

[nielsle]

Some users will be confused that a postfix macro can change the control flow even though it looks like a method. This can lead to obfuscated code. I can imagine debugging code where my_log.debug!("x {:?}",x) sometimes returns an Err(Error).

But the feature also looks very useful and simple.

[joshtriplett]

@nielsle

Some users will be confused that a postfix macro can change the control flow even though it looks like a method.

Non-postfix macros can do the same. In both cases, I think the ! in the macro name calls sufficient attention to the possibility that it might do something unusual. And prospective macro writers should use caution when writing control-flow macros to avoid surprising behavior.

I can imagine debugging code where my_log.debug!("x {:?}",x) sometimes returns an Err(Error).

I definitely wouldn't expect that to happen, any more than I'd expect debug!("x {:?}", x); to sometimes return from the calling function.

But the feature also looks very useful and simple.

Thanks!

[d4h0]

@satvikpendem: I don't think it would be wise to let postfix macros alter their left-hand expression (and I would be pretty astonished if people who decide if postfix macros are implemented think differently).

I think, the left-hand should have type expr, ident, etc., so that postfix macros can use what is to their left (i.e., no tt), but not alter it (I have never used procedural macros, so I'm not sure, if that would be even possible there).

[d4h0]

As @NyxCode mentioned in the postfix match issue:

Would it be possible to add postfix macros on unstable, to be able to experiment with proposals like postfix match?

.await, ?, and the method chain syntax clearly demonstrate that postfix-based syntaxes can be beneficial. Therefore, it would be advantageous if experimentation with such syntax constructs is made easier on unstable.

Even if postfix macros are never stabilized, they would still be highly useful for such experimentation (and help avoid potential mistakes).

[recatek]

Currently working on (yet another) ECS library in Rust that's heavily generic- and macro-based. I would love to be able to do

world.ecs_query!(component_a, component_b, component_c)

instead of

ecs_query!(world, component_a, component_b, component_c)

especially when this query is part of a long chain.

[daniel-pfeiffer]

@kevinushey Before I read this, on the contrary I was assuming that chained macros would be members. I come from a place where I need to add methods that can't currently be expressed as functions. The same goes for object.unwrap_or! { ... } which would only exist for Option and Result.

Maybe we can have it both ways, if this is feasible without rewriting the parser: when there are member macros they take precedence over free macros. And they apply only where implemented. With extended 2.0 syntax:

impl X {
    macro x0($self) { todo!() }
    macro x1($self, $x:expr) { todo!() }
    macro xn($self, ..) { // additional params below
        () => { todo!() },
        ($x:expr) => { todo!() },
        ($x1:ident $x2:tt) => { todo!() },
    }
}
trait Y {
    macro y0($self) { todo!() }
    macro y1($self, $y:expr);
    macro yn($self, ..); // additional params in impl
}

[oli-obk]

I can say with confidence that there is no support for this in T-types.

Furthermore we don't have the infrastructure in the compiler for supporting this, and are not expecting this to be possible within the next 5-10 years. There is some serious reengineering required to get there, and the incremental steps required for it have stalled two years ago.

[oli-obk]

edit: wrong text section highlighted, so read the comment as a free comment

I think this RFC needs to be evaluated on the basis that such a system will never come to Rust, as that is the likeliest situation we'll find ourselves in afaict.

If this RFC is only accepted because it leaves the door open to a future type based extension, then it should not be accepted imo.

If this section causes opponents of the RFC to accept the RFC as a compromise, because a type based system is expected to come in the future, then this should be rediscussed.

Imo this RFC should explicitly state that we will never get a type based system, and include T-types in the FCP.

[oli-obk]

Why can't we just use &$self as an expression and have that evaluate to the tokens that were passed as the self? Just like with other arguments, if you don't want it evaluated twice, first evaluate it into a let binding, then use that twice.

[SpriteOvO]

I'm suddenly looking forward to this feature, because I find it seems useful for simplifying optional parameters in macros.

In log crate, a full use example of a logging macro looks like this:

info!(target: "telemetry", username, ip, options = login_options; "user login to {}", platform);
//                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ key-values

Some other log crates may provide more optional parameters like the target here. If there is postfix macro support, this will make the code look super clean, and the parameters could be unordered.

info!("user login to {}", platform).target!("telemetry").kv!(username, ip, options = login_options);

[workingjubilee]

@rust-lang/lang I would like to propose that you rfcbot fcp close or rfcbot fcp postpone this. It is a very old RFC from 2018. It seems to have a semi-favorable reception, but it should be set aside and revisited by someone with the bandwidth to incorporate all the feedback and drive it anew.

[michalfita]

Here's a practical example where this would be useful: ros2-rust/ros2_rust#460 or https://github.com/ros2-rust/ros2_rust/pull/459/files.

It would be much nicer to be able to call node.info().log!("Format {}", argument).

[jplatte]

@joshtriplett do you plan on getting back to this? If not, I'd be interested in making a new RFC based on this + comments here that haven't yet been incorporated.

[traviscross]

The proposal I'd personally like to see would be for what I've been calling "simplest postfix macros". The macro matcher's first fragment would be $self:expr -- the fragment specifier would be required to be expr.¹ The macro would control evaluation of that expression as usual.

This would eliminate much of the complexity of this RFC and its expressiveness limitations. I and others had concerns about those both (e.g.), and it was these kind of concerns that stalled it out. Of course, there remain other concerns about whether it would be too surprising, from a user perspective, to give the macro this evaluation control. Personally, I think that'd be OK, but I know some have concerns there too, and that would need to be discussed at length in any proposal along these lines.

Footnotes

1. $self is already a valid fragment identifier, so we'd need to signal that the macro matcher is a postfix one in some other way, e.g. with an attribute or with some other syntax (and/or reclaim $self for this purpose over an edition). ↩

[Nadrieril]

I've been thinking about this in the context of the Field Projections proposal, and this k#autoref thing has a fundamental limitation that boils down to: "postfix things want to act on places, and trying to use a reference/ptr instead of a place is restrictive".

1. Assignment doesn't work:

macro_rules! write {
    ($self:self, $val:expr) => ({
        $self = $val;
        $self
    })
}
let mut x;
let _ = x.write!(Some(42)).take();

Here if the macro captures &mut x we'd get a borrowck error, despite the fact that the obvious token substitution works.

Also note how returning the $self is also an issue: for the k#autoref to make sense it can't change the type of $self, so I imagine the actual desugaring would be like for closures: we capture captured_x = &mut x, and then use *captured_x in place of $self everywhere. But this prevents returning $self like I'm doing here since that would cause a move (or here a very surprising copy).

2. Custom ptr autoref:

macro_rules! call {
    ($self:self, $name:ident, $($args:expr),*) => ({
        $self.$name($($args),*)
    })
}
let x: Rc<Foo> = ...;
// Assume `method` takes `RcRef<Self>`:
x.a.method(); // the goal of field projections is to make this work
x.a.call!(method); // for this to work, `k#autoref` should infer the right custom pointer

It's not impossible to infer the right custom pointer here since there's a single call and autoref has to know how to do that, but for multiple ones we get in trouble:

macro_rules! do_two_things {
    ($self:self) => ({
        // Assume `method` takes `RcRef<Self>` and `other_method` takes `&self`.
        $self.method();
        $self.other_method();
    })
}
let x: Rc<Foo> = ...;
x.a.do_two_things(); // there's no "most general type" with which to autoref `x.a` here.

[joshtriplett]

Talked to @Nadrieril in detail about this on a call. More details soon, but one key insight Nadri pointed out: closure capture was the wrong model for autoref, and the right model is something more like method receivers. That would also solve case (2).

[joshtriplett]

@jplatte wrote:

@joshtriplett do you plan on getting back to this? If not, I'd be interested in making a new RFC based on this + comments here that haven't yet been incorporated.

Currently getting back to this together with @Nadrieril.

[Nadrieril]

That's a lot of pressure 🙈. Let's chat @jplatte, curious to hear your take!

[Nadrieril]

Having indeed chatted with Josh, my feeling is nuanced about this proposal. One important ergonomic requirement is that users should never have to write (&mut <something>).macro!() to get typechecking to work; I could see this being harder with the "just paste the tokens" proposal. I might write some more detailed thoughts if I manage to get them in order.

What's clear to me regardless is that we should start an experiment to implement the just-paste-tokens proposal, so we can get a feel for it and see what ppl come up with to inform further proposals.

[dianne]

Even with match, I don't think the temporary lifetimes for arguments would quite be the same as with normal method calls, since match arms are temporary scopes. For example, in

macro_rules! wrap_method {
    ($self:self, $arg:expr) => { self.method($arg) }
}

the temp() in receiver.wrap_method!(&temp()) would have a shorter lifetime than the temp() in receiver.method(&temp()); the former would be dropped at the end of the match arm in the desugared macro expansion, whereas the latter would be dropped at the end of the enclosing temporary scope (allowing it to be used in method chains, e.g.). This could be worked around with new scoping constructs, though. For a very direct solution, there could e.g. be a way to mark match arms as not being temporary scopes. For something less purpose-built, it could maybe use super let to scope the macro expansion body's temporaries as if it was written in place of the match¹.

Maybe also of note: wrapping postfix macro expansions in match means they can only expand to value expressions. This is fine for method-call-like macros and postfix .match!, since those both represent values, but unfortunate for field/projection-like macros. x.proj!() wouldn't be usable as a place alias; instead, it would evaluate to a temporary when used in a place expression context.

Similarly, although postfix macros could be written to allow their arguments on the right of the macro invocation to participate in temporary lifetime extension, there's no direct way to allow the "receiver" to be lifetime-extended. This is fine for method-call-like macros (which don't need lifetime extension) and postfix .match! (which only needs to lifetime-extend its arms), but for field/projection-like macros, it could be nice to to have the temporary lifetime extension behavior projection expressions do: in an extending context, &temp().field lifetime-extends the temp() being projected from, so ideally I think it should be possible to write a proj! macro such that &temp().proj!() would also lifetime-extend the temp().

Footnotes

1. I think super let as implemented today may not quite be able to express this, but in my opinion it's something it should be able to do. Temporary lifetime extension for blocks rust#146098 e.g. would be one way of getting the missing functionality. ↩