Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Jun 10, 2022

• Internal change

  Show description for d229a44
  

  Mesh TensorFlow Team

  committed
  d229a44

  Copy full SHA for d229a44

Commits on May 12, 2022

• Make mesh_tensorflow's call of `get_replicated_var_handle` backward-compatible with tf <= 2.8.0. Fixes https://github.com/google-research/text-to-text-transfer-transformer/issues/1020.

  Show description for 6d44045
  

  adarob

  authored and

  Mesh TensorFlow Team

  committed
  6d44045

  Copy full SHA for 6d44045

Commits on May 5, 2022

• bump version number to release updated PyPI package that includes last year enhancements

  Show description for b631ad5
  

  aboSamoor

  authored and

  Mesh TensorFlow Team

  committed
  b631ad5

  Copy full SHA for b631ad5

Commits on Mar 25, 2022

• Explicitly import estimator from tensorflow as a separate import instead of accessing it via tf.estimator and depend on the tensorflow estimator target.

  Show description for 6b31c0f
  

  qlzh727

  authored and

  Mesh TensorFlow Team

  committed
  6b31c0f

  Copy full SHA for 6b31c0f

Commits on Mar 24, 2022

• Remove unused comments related to Python 2 compatibility.

  Show description for 58153bf
  

  yilei

  authored and

  Mesh TensorFlow Team

  committed
  58153bf

  Copy full SHA for 58153bf

Commits on Mar 21, 2022

• Make TPU variable name deterministic.

  Show description for cccc00e
  

  chuanhaozhuge

  authored and

  Mesh TensorFlow Team

  committed
  cccc00e

  Copy full SHA for cccc00e

Commits on Feb 18, 2022

• #HyperPrompt Part 2 of HyperPrompt implementation: the actual computation of HyperPrompt inside self-attention layer.

  Show description for 39f4bd6
  

  Mesh TensorFlow Team

  committed
  39f4bd6

  Copy full SHA for 39f4bd6

Commits on Jan 26, 2022

• Use math.gcd instead of fractions.gcd, the former is deprecated in Python 3.5 and removed in 3.9.

  Show description for bbb6ce7
  

  yilei

  authored and

  Mesh TensorFlow Team

  committed
  bbb6ce7

  Copy full SHA for bbb6ce7

• Split out optimizer call for internal purposes.

  Show description for ead60ee
  

  William Fedus

  authored and

  Mesh TensorFlow Team

  committed
  ead60ee

  Copy full SHA for ead60ee

Commits on Jan 19, 2022

• fix typo in logging statement.

  Show description for a32810e
  

  Mesh TensorFlow Team

  committed
  a32810e

  Copy full SHA for a32810e

Commits on Oct 18, 2021

• Make sure gates are not normalized for n=1 for top_n routing

  Show description for 57ed401
  

  Mesh TensorFlow Team

  committed
  57ed401

  Copy full SHA for 57ed401

Commits on Oct 6, 2021

• Add a link to the Primer paper

  Show description for 52a2332
  

  Mesh TensorFlow Team

  committed
  52a2332

  Copy full SHA for 52a2332

Commits on Sep 14, 2021

• Add utility to save score predictions to TFRecords for scoring large datasets.

  Show description for cfc7a67
  

  Hyperparticle

  authored and

  Mesh TensorFlow Team

  committed
  cfc7a67

  Copy full SHA for cfc7a67

Commits on Sep 12, 2021

• Remove unnecessary name and cwise in squared relu.

  Show description for f08b18e
  

  David So

  authored and

  Mesh TensorFlow Team

  committed
  f08b18e

  Copy full SHA for f08b18e

Commits on Sep 10, 2021

• Squared ReLU from Primer paper.

  Show description for 739fb09
  

  David So

  authored and

  Mesh TensorFlow Team

  committed
  739fb09

  Copy full SHA for 739fb09

Commits on Sep 8, 2021

• Expert Attention Fixes:

  Show description for 4758510
  

  Mesh TensorFlow Team

  committed
  4758510

  Copy full SHA for 4758510

• Internal

  Show description for acf6247
  

  Mesh TensorFlow Team

  committed
  acf6247

  Copy full SHA for acf6247

Commits on Sep 7, 2021

• Expert Attention Fixes:

  Show description for 69bd9c7
  

  Mesh TensorFlow Team

  committed
  69bd9c7

  Copy full SHA for 69bd9c7

Commits on Aug 24, 2021

• Add more extensive top-2 logging.

  Show description for 21c4ef3
  

  Mesh TensorFlow Team

  committed
  21c4ef3

  Copy full SHA for 21c4ef3

Commits on Aug 20, 2021

• Remove dataset checkpoint policy override now that b/181765832 is resolved.

  Show description for 1f381ce
  

  adarob

  authored and

  Mesh TensorFlow Team

  committed
  1f381ce

  Copy full SHA for 1f381ce

Commits on Aug 18, 2021

• heterogeneous mixture of experts layer

  Show description for b4810b6
  

  Mesh TensorFlow Team

  committed
  b4810b6

  Copy full SHA for b4810b6

• Only add z_loss to losses if during training.

  Show description for 2a6fa4c
  

  Mesh TensorFlow Team

  committed
  2a6fa4c

  Copy full SHA for 2a6fa4c

Commits on Aug 16, 2021

• Fix bug in shared_kv attention for autoregressive decoding.

  Show description for 0a212ca
  

  Mesh TensorFlow Team

  committed
  0a212ca

  Copy full SHA for 0a212ca

• Change second d_model_split dim's size to be the output shape, instead of input shape. This allows it to work for layers where the input size is different than the output size.

  Show description for 7c09bb9
  

  Mesh TensorFlow Team

  committed
  7c09bb9

  Copy full SHA for 7c09bb9

Commits on Aug 11, 2021

• Update mesh tensorflow to use device assignments to map logical to physical processor numbers on N-D Meshes. Currently only enabled when logical cores per replica is set to 1.

  Show description for 3922a8f
  

  Mesh TensorFlow Team

  committed
  3922a8f

  Copy full SHA for 3922a8f

Commits on Aug 10, 2021

• Add more options to Experts Attention. These options remove 1/3 of the all2all communication costs:

  Show description for ef0cb24
  

  Mesh TensorFlow Team

  committed
  ef0cb24

  Copy full SHA for ef0cb24

Commits on Aug 3, 2021

• Add in Z-loss to all routing algorithms.

  Show description for 3aaa765
  

  Mesh TensorFlow Team

  committed
  3aaa765

  Copy full SHA for 3aaa765

Commits on Aug 2, 2021

• Minor changes to make Experts Attention work.

  Show description for d419b67
  

  Mesh TensorFlow Team

  committed
  d419b67

  Copy full SHA for d419b67

Commits on Jul 29, 2021

• Add top-n routing, which generalized top-2 routing. Improves model quality for larger capacity factors (e.g. 2.0+).

  Show description for 7e78cf8
  

  Mesh TensorFlow Team

  committed
  7e78cf8

  Copy full SHA for 7e78cf8

Commits on Jul 27, 2021

• Add z_loss on all attention logits. This does not change model quality and can effectively decrease the attention logits by order of magnitude.

  Show description for b0d78f4
  

  Mesh TensorFlow Team

  committed
  b0d78f4

  Copy full SHA for b0d78f4

Commits on Jul 26, 2021

• Using the soft loss dtype instead of hardcoding bfloat16.

  Show description for 8853284
  

  Mesh TensorFlow Team

  committed
  8853284

  Copy full SHA for 8853284

Commits on Jul 21, 2021

• - Fix casting for NTLB.

  Show description for d3ffc32
  

  Mesh TensorFlow Team

  committed
  d3ffc32

  Copy full SHA for d3ffc32

• Next gen fish optimizations for MeshTF.

  Show description for 1f5d35c
  

  Mesh TensorFlow Team

  committed
  1f5d35c

  Copy full SHA for 1f5d35c

Commits on Jul 19, 2021

• Switch logging to warm to not fail when using deterministic dataset checkpointing.

  Show description for 38e58c4
  

  Mesh TensorFlow Team

  committed
  38e58c4

  Copy full SHA for 38e58c4

Commits on Jul 16, 2021

• Add z-loss to the top_2_gating method.

  Show description for f594159
  

  Mesh TensorFlow Team

  committed
  f594159

  Copy full SHA for f594159