Alex Rivera | Logout

What is a good heuristic for determining the tab width used in a source file?

Asked 2011-08-04T10:37:37.147
12

I would like to determine the tab width used in source files indented with spaces. This is not hard for files with particularly regular indentation, where the leading spaces are only used for indentation, always in multiples of the tab width, and with indentation increasing one level at at time. But many files will have some departure from this sort of regular indentation, generally for some form of vertical alignment. I'm thus looking for a good heuristic to estimate what tab width was used, allowing some possibility for irregular indentation.

The motivation for this is writing an extension for the SubEthaEdit editor. SubEthaEdit unfortunately doesn't make the tab width available for scripting, so I'm going to guess at it based on the text.

A suitable heuristic should:

  • Perform well enough for interactive use. I don't imagine this will be a problem, and just a portion of the text can be used if need be.
  • Be language independent.
  • Return the longest suitable tab width. For example, any file with a tab width of four spaces could also be a file with two-space tabs, if every indentation was actually by twice as many levels. Clearly, four spaces would be the right choice.
  • Always get it right if the indentation is completely regular.

Some simplifying factors:

  • At least one line can be assumed to be indented.
  • The tab width can be assumed to be at least two spaces.
  • It's safe to assume that indentation is done with spaces only. It's not that I have anything against tabs---quite the contrary, I'll check first if there are any tabs used for indentation and handle it separately. This does mean that indentation mixing tabs and spaces might not be handled properly, but I don't consider it important.
  • It may be assumed that there are no lines containing only whitespace.
  • Not all languages need to be handled correctly. For example, success or failure with
language-agnostic indentation heuristics
Edit
Report

1 Answer

1

Your choices are (realistically) 2,3,4,5,6,7,8.

I'd scan the the first 50-100 lines or so using something like what @FastAl suggested. I'd probably lean toward just blindly pulling the spaces count from the front of any row with text and counting the length of the white space string. Left trimming lines and running length twice seems like a waste if you have regex available. Also, I'd do System.Math.abs(indent - previndent) so you get de-indent data. The regex would be this:

row.matches('^( +)[^ ]') # grab all the spaces from line start to non-space.

Once you've got a statistic for which of the 7 options has the highest count, run with it as the first guess. For 8, 6, and 4 you should check to see if there is also a significant count (2nd place or over 10% or some other cheapo heuristic) for 4 and 2, 3, or 2. If there are a lot of 12s (or 9s) that might hint that 4 (or 3) is a better choice than 8 (or 6) as well. Dropping or adding more than 2 levels at a time (usually collapsed ending brackets) is super rare.

Irrelevant mumbling

The one problem I see is that old .c code in particular has this nasty pattern going on in it:

code level 0
/* Fancy comments get weird spacing because there 
 * is an extra space beyond the *
 * looks like one space!
 */
  code indent (2 spaces)
  /* Fancy comments get weird spacing because there 
   * is an extra space beyond the *
   * looks like three spaces!
   */

code level 0
  code indent (2 spaces)
  /* comment at indent level 1
     With no stars you wind up with 2 spaces + 3 spaces.
  */

Yuck. I don't know how you deal with comment standards like that. For code that is "c" like you might have to deal with comments special in version 2.0... but I would just ignore it for now.

Your final issue is dealing with lines that don't match your assumptions. My suggestion would be to "tab" them to depth and t

answered 2011-08-24T19:01:11.057

Your Answer