Split MacArgs in two.

`MacArgs` is an enum with three variants: `Empty`, `Delimited`, and `Eq`. It's
used in two ways:
- For representing attribute macro arguments (e.g. in `AttrItem`), where all
  three variants are used.
- For representing function-like macros (e.g. in `MacCall` and `MacroDef`),
  where only the `Delimited` variant is used.

In other words, `MacArgs` is used in two quite different places due to them
having partial overlap. I find this makes the code hard to read. It also leads
to various unreachable code paths, and allows invalid values (such as
accidentally using `MacArgs::Empty` in a `MacCall`).

This commit splits `MacArgs` in two:
- `DelimArgs` is a new struct just for the "delimited arguments" case. It is
  now used in `MacCall` and `MacroDef`.
- `AttrArgs` is a renaming of the old `MacArgs` enum for the attribute macro
  case. Its `Delimited` variant now contains a `DelimArgs`.

Various other related things are renamed as well.

These changes make the code clearer, avoids several unreachable paths, and
disallows the invalid values.

Author: nnethercote

src/expr.rs

@@ -1341,7 +1341,7 @@ pub(crate) fn can_be_overflowed_expr(
         }
         ast::ExprKind::MacCall(ref mac) => {
             match (
-                rustc_ast::ast::MacDelimiter::from_token(mac.args.delim().unwrap()),
+                rustc_ast::ast::MacDelimiter::from_token(mac.args.delim.to_token()),
                 context.config.overflow_delimited_expr(),
             ) {
                 (Some(ast::MacDelimiter::Bracket), true)

src/macros.rs

@@ -208,7 +208,7 @@ fn rewrite_macro_inner(
         original_style
     };
 
-    let ts = mac.args.inner_tokens();
+    let ts = mac.args.tokens.clone();
     let has_comment = contains_comment(context.snippet(mac.span()));
     if ts.is_empty() && !has_comment {
         return match style {
@@ -392,7 +392,7 @@ pub(crate) fn rewrite_macro_def(
         return snippet;
     }
 
-    let ts = def.body.inner_tokens();
+    let ts = def.body.tokens.clone();
     let mut parser = MacroParser::new(ts.into_trees());
     let parsed_def = match parser.parse() {
         Some(def) => def,
@@ -1087,7 +1087,7 @@ pub(crate) fn convert_try_mac(
 ) -> Option<ast::Expr> {
     let path = &pprust::path_to_string(&mac.path);
     if path == "try" || path == "r#try" {
-        let ts = mac.args.inner_tokens();
+        let ts = mac.args.tokens.clone();
 
         Some(ast::Expr {
             id: ast::NodeId::root(), // dummy value

src/parse/macros/asm.rs

@@ -5,7 +5,7 @@ use crate::rewrite::RewriteContext;
 
 #[allow(dead_code)]
 pub(crate) fn parse_asm(context: &RewriteContext<'_>, mac: &ast::MacCall) -> Option<AsmArgs> {
-    let ts = mac.args.inner_tokens();
+    let ts = mac.args.tokens.clone();
     let mut parser = super::build_parser(context, ts);
     parse_asm_args(&mut parser, context.parse_sess.inner(), mac.span(), false).ok()
 }

src/parse/macros/cfg_if.rs

@@ -23,7 +23,7 @@ fn parse_cfg_if_inner<'a>(
     sess: &'a ParseSess,
     mac: &'a ast::MacCall,
 ) -> Result<Vec<ast::Item>, &'static str> {
-    let ts = mac.args.inner_tokens();
+    let ts = mac.args.tokens.clone();
     let mut parser = build_stream_parser(sess.inner(), ts);
 
     let mut items = vec![];