Specifics of usage and internal work of *Set* functions

Question

I just noticed one undocumented feature of internal work of *Set* functions in Mathematica.

Consider:

In[1]:= a := (Print["!"]; a =.; 5);
a[b] = 2;
DownValues[a]

During evaluation of In[1]:= !

Out[3]= {HoldPattern[a[b]] :> 2}

but

In[4]:= a := (Print["!"]; a =.; 5);
a[1] = 2;
DownValues[a]

During evaluation of In[4]:= !

During evaluation of In[4]:= Set::write: Tag Integer in 5[1] is Protected. >>

Out[6]= {HoldPattern[a[b]] :> 2}

What is the reason for this difference? Why a is evaluated although Set has attribute HoldFirst? For which purposes such behavior is useful?

And note also this case:

In[7]:= a := (Print["!"]; a =.; 5)
a[b] ^= 2
UpValues[b]
a[b]

During evaluation of In[7]:= !

Out[8]= 2

Out[9]= {HoldPattern[5[b]] :> 2}

Out[10]= 2

As you see, we get the working definition for 5[b] avoiding Protected attribute of the tag Integer which causes error in usual cases:

In[13]:= 5[b] = 1

During evaluation of In[13]:= Set::write: Tag Integer in 5[b] is Protected. >>

Out[13]= 1

The other way to avoid this error is to use TagSet*:

In[15]:= b /: 5[b] = 1
UpValues[b]

Out[15]= 1

Out[16]= {HoldPattern[5[b]] :> 1}

Why are these features?

---

Regarding my question why we can write a := (a =.; 5); a[b] = 2 while cannot a := (a =.; 5); a[1] = 2. In really in Mathematica 5 we cannot write a := (a =.; 5); a[b] = 2 too:

In[1]:=
a:=(a=.;5);a[b]=2
From In[1]:= Set::write: Tag Integer in 5[b] is Protected. More...
Out[1]=
2

(The above is copied from Mathematica 5.2)

We can see what happens internally in new versions of Mathematica when we evaluate a := (a =.; 5); a[b] = 2:

In[1]:= a:=(a=.;5);
Trace[a[b]=2,TraceOriginal->True]
Out[2]= {a[b]=2,{Set},{2},a[b]=2,{With[{JLink`Private`obj$=a},RuleCondition[$ConditionHold[$ConditionHold[JLink`CallJava`Private`setField[JLink`Private`obj$[b],2]]],Head[JLink`Private`obj$]===Symbol&&StringMatchQ[Context[JLink`Private`obj$],JLink`Objects`*]]],{With},With[{JLink`Private`obj$=a},RuleCondition[$ConditionHold[$ConditionHold[JLink`CallJava`Private`setField[JLink`Private`obj$[b],2]]],Head[JLink`Private`obj$]===Symbol&&StringMatchQ[Context[JLink`Private`obj$],JLink`Objects`*]]],{a,a=.;5,{CompoundExpression},a=.;5,{a=.,{Unset},a=.,Null},{5},5},RuleCondition[$ConditionHold[$ConditionHold[JLink`CallJava`Private`setField[5[b],2]]],Head[5]===Symbol&&StringMatchQ[Context[5],JLink`Objects`*]],{RuleCondition},{Head[5]===Symbol&&StringMatchQ[Context[5],JLink`Objects`*],{And},Head[5]===Symbol&&StringMatchQ[Context[5],JLink`Objects`*],{Head[5]===Symbol,{SameQ},{Head[5],{Head},{5},Head[5],Integer},{Symbol},Integer===Symbol,False},False},RuleCondition[$ConditionHold[$ConditionHold[JLink`CallJava`Private`setField[5[b],2]]],False],Fail},a[b]=2,{a[b],{a},{b},a[b]},2}

I was very surprised to see calls to Java in such a pure language-related operation as assigning a value to a variable. Is it reasonable to use Java for such operations at all?

---

Todd Gayley (Wolfram Research) has explained this behavior:

At the start, let me point out that in Mathematica 8, J/Link no longer overloads Set. An internal kernel mechanism was created that, among other things, allows J/Link to avoid the need for special, er, "tricks" with Set.

J/Link has overloaded Set from the very beginning, almost twelve years ago. This allows it support this syntax for assigning a value to a Java field:

 javaObject@field = value

The overloaded definition of Set causes a slowdown in assignments of the form

 _Symbol[_Symbol] = value

Of course, assignment is a fast operation, so the slowdown is small in real terms. Only highly specialized types of programs are likely to be significantly affected.

The Set overload does not cause a call to Java on assignments that do not involve Java objects (this would be very costly). This can be verified with a simple use of TracePrint on your a[b]=c.

It does, as you note, make a slight change in the behavior of assignments that match _Symbol[_Symbol] = value. Specifically, in f[_Symbol] = value, f gets evaluated twice. This can cause problems for code with the following (highly unusual) form:

 f := SomeProgramWithSideEffects[]
 f[x] = 42

I cannot recall ever seeing "real" code like this, or seeing a problem reported by a user.

This is all moot now in 8.0.

Answer 1

Taking the case of UpSet first, this is expected behavior. One can write:

 5[b] ^= 1

The assignment is made to b not the Integer 5.

Regarding Set and SetDelayed, while these have Hold attributes, they still internally evaluate expressions. This allows things such as:

p = n : (_List | _Integer | All);

f[p] := g[n]

Test:

f[25]
f[{0.1, 0.2, 0.3}]
f[All]

   g[25]

   g[{0.1, 0.2, 0.3}]

   g[All]

One can see that heads area also evaluated. This is useful at least for UpSet:

p2 = head : (ff | gg);
p2[x] ^:= Print["Echo ", head];

ff[x]
gg[x]

Echo ff

Echo gg

It is easy to see that it happens also with Set, but less clear to me how this would be useful:

j = k;
j[5] = 3;
DownValues[k]

(* Out=  {HoldPattern[k[5]] :> 3}  *)

---

My analysis of the first part of your question was wrong. I cannot at the moment see why a[b] = 2 is accepted and a[1] = 2 is not. Perhaps at some stage of assignment the second one appears as 5[1] = 2 and a pattern check sets off an error because there are no Symbols on the LHS.

Answer 2

The behavour you show appears to be a bug in 7.0.1 (and possibly earlier) that was fixed in Mathematica 8. In Mathematica 8, both of your original a[b] = 2 and a[1] = 2 examples give the Set::write ... is protected error.

The problem appears to stem from the JLink-related down-value of Set that you identified. That rule implements the JLink syntax used to assign a value to the field of a Java object, e.g. object@field = value.

Set in Mathematica 8 does not have that definition. We can forcibly re-add a similar definition, thus:

Unprotect[Set]
HoldPattern[sym_Symbol[arg_Symbol]=val_] :=
  With[{obj=sym}
  , setField[obj[arg], val] /; Head[obj] === Symbol && StringMatchQ[Context[obj],"Something`*"]
  ]

After installing this definition in Mathematica 8, it now exhibits the same inconsistent behaviour as in Mathematica 7.

I presume that JLink object field assignment is now accomplished through some other means. The problematic rule looks like it potentially adds costly Head and StringMatchQ tests to every evaluation of the form a[b] = .... Good riddance?